{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "s_qNSzzyaCbD"
      },
      "source": [
        "##### Copyright 2019 The TensorFlow Authors."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 0,
      "metadata": {
        "cellView": "form",
        "colab": {},
        "colab_type": "code",
        "id": "jmjh290raIky"
      },
      "outputs": [],
      "source": [
        "#@title Licensed under the Apache License, Version 2.0 (the \"License\");\n",
        "# you may not use this file except in compliance with the License.\n",
        "# You may obtain a copy of the License at\n",
        "#\n",
        "# https://www.apache.org/licenses/LICENSE-2.0\n",
        "#\n",
        "# Unless required by applicable law or agreed to in writing, software\n",
        "# distributed under the License is distributed on an \"AS IS\" BASIS,\n",
        "# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n",
        "# See the License for the specific language governing permissions and\n",
        "# limitations under the License."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "J0Qjg6vuaHNt"
      },
      "source": [
        "# Transformer model for language understanding"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "AOpGoE2T-YXS"
      },
      "source": [
        "\u003ctable class=\"tfo-notebook-buttons\" align=\"left\"\u003e\n",
        "  \u003ctd\u003e\n",
        "    \u003ca target=\"_blank\" href=\"https://www.tensorflow.org/alpha/tutorials/text/transformer\"\u003e\n",
        "    \u003cimg src=\"https://www.tensorflow.org/images/tf_logo_32px.png\" /\u003e\n",
        "    View on TensorFlow.org\u003c/a\u003e\n",
        "  \u003c/td\u003e\n",
        "  \u003ctd\u003e\n",
        "    \u003ca target=\"_blank\" href=\"https://colab.research.google.com/github/tensorflow/docs/blob/master/site/en/r2/tutorials/text/transformer.ipynb\"\u003e\n",
        "    \u003cimg src=\"https://www.tensorflow.org/images/colab_logo_32px.png\" /\u003e\n",
        "    Run in Google Colab\u003c/a\u003e\n",
        "  \u003c/td\u003e\n",
        "  \u003ctd\u003e\n",
        "    \u003ca target=\"_blank\" href=\"https://github.com/tensorflow/docs/blob/master/site/en/r2/tutorials/text/transformer.ipynb\"\u003e\n",
        "    \u003cimg src=\"https://www.tensorflow.org/images/GitHub-Mark-32px.png\" /\u003e\n",
        "    View source on GitHub\u003c/a\u003e\n",
        "  \u003c/td\u003e\n",
        "\u003c/table\u003e"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "M-f8TnGpE_ex"
      },
      "source": [
        "This tutorial trains a \u003ca href=\"https://arxiv.org/abs/1706.03762\" class=\"external\"\u003eTransformer model\u003c/a\u003e to translate Portuguese to English. This is an advanced example that assumes knowledge of [text generation](text_generation.ipynb) and [attention](nmt_with_attention.ipynb).\n",
        "\n",
        "The core idea behind the Transformer model is *self-attention*—the ability to attend to different positions of the input sequence to compute a representation of that sequence. Transformer creates stacks of self-attention layers and is explained below in the sections *Scaled dot product attention* and *Multi-head attention*.\n",
        "\n",
        "A transformer model handles variable-sized input using stacks of self-attention layers instead of [RNNs](text_classification_rnn.ipynb) or [CNNs](../images/intro_to_cnns.ipynb). This general architecture has a number of advantages:\n",
        "\n",
        "* It make no assumptions about the temporal/spatial relationships across the data. This is ideal for processing a set of objects (for example, [StarCraft units](https://deepmind.com/blog/alphastar-mastering-real-time-strategy-game-starcraft-ii/#block-8)).\n",
        "* Layer outputs can be calculated in parallel, instead of a series like an RNN.\n",
        "* Distant items can affect each other's output without passing through many RNN-steps, or convolution layers (see [Scene Memory Transformer](https://arxiv.org/pdf/1903.03878.pdf) for example).\n",
        "* It can learn long-range dependencies. This is a challenge in many sequence tasks.\n",
        "\n",
        "The downsides of this architecture are:\n",
        "\n",
        "* For a time-series, the output for a time-step is calculated from the *entire history* instead of only the inputs and current hidden-state. This _may_ be less efficient.   \n",
        "* If the input *does* have a  temporal/spatial relationship, like text, some positional encoding must be added or the model will effectively see a bag of words. \n",
        "\n",
        "After training the model in this notebook, you will be able to input a Portuguese sentence and return the English translation.\n",
        "\n",
        "\u003cimg src=\"https://www.tensorflow.org/images/tutorials/transformer/attention_map_portuguese.png\" width=\"800\" alt=\"Attention heatmap\"\u003e"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 0,
      "metadata": {
        "colab": {},
        "colab_type": "code",
        "id": "V_sMDzDT9x4r"
      },
      "outputs": [],
      "source": [
        "from __future__ import absolute_import, division, print_function, unicode_literals\n",
        "\n",
        "!pip install tf-nightly-gpu-2.0-preview\n",
        "import tensorflow_datasets as tfds\n",
        "import tensorflow as tf\n",
        "\n",
        "import time\n",
        "import numpy as np\n",
        "import matplotlib.pyplot as plt"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "fd1NWMxjfsDd"
      },
      "source": [
        "## Setup input pipeline"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "t4_Qt8W1hJE_"
      },
      "source": [
        "Use [TFDS](https://www.tensorflow.org/datasets) to load the [Portugese-English translation dataset](https://github.com/neulab/word-embeddings-for-nmt) from the [TED Talks Open Translation Project](https://www.ted.com/participate/translate).\n",
        "\n",
        "This dataset contains approximately 50000 training examples, 1100 validation examples, and 2000 test examples."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 0,
      "metadata": {
        "colab": {},
        "colab_type": "code",
        "id": "8q9t4FmN96eN"
      },
      "outputs": [],
      "source": [
        "examples, metadata = tfds.load('ted_hrlr_translate/pt_to_en', with_info=True,\n",
        "                               as_supervised=True)\n",
        "train_examples, val_examples = examples['train'], examples['validation']"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "RCEKotqosGfq"
      },
      "source": [
        "Create a custom subwords tokenizer from the training dataset. "
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 0,
      "metadata": {
        "colab": {},
        "colab_type": "code",
        "id": "KVBg5Q8tBk5z"
      },
      "outputs": [],
      "source": [
        "tokenizer_en = tfds.features.text.SubwordTextEncoder.build_from_corpus(\n",
        "    (en.numpy() for pt, en in train_examples), target_vocab_size=2**13)\n",
        "\n",
        "tokenizer_pt = tfds.features.text.SubwordTextEncoder.build_from_corpus(\n",
        "    (pt.numpy() for pt, en in train_examples), target_vocab_size=2**13)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 0,
      "metadata": {
        "colab": {
          "height": 53
        },
        "colab_type": "code",
        "id": "4DYWukNFkGQN",
        "outputId": "fce46452-9b01-4330-e2b8-2777f9102a7f"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Tokenized string is [7915, 1248, 7946, 7194, 13, 2799, 7877]\n",
            "The original string: Transformer is awesome.\n"
          ]
        }
      ],
      "source": [
        "sample_string = 'Transformer is awesome.'\n",
        "\n",
        "tokenized_string = tokenizer_en.encode(sample_string)\n",
        "print ('Tokenized string is {}'.format(tokenized_string))\n",
        "\n",
        "original_string = tokenizer_en.decode(tokenized_string)\n",
        "print ('The original string: {}'.format(original_string))\n",
        "\n",
        "assert original_string == sample_string"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "o9KJWJjrsZ4Y"
      },
      "source": [
        "The tokenizer encodes the string by breaking it into subwords if the word is not in its dictionary."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 0,
      "metadata": {
        "colab": {
          "height": 143
        },
        "colab_type": "code",
        "id": "bf2ntBxjkqK6",
        "outputId": "7fe624d6-5155-4b7b-9bec-3e4bc0b3127b"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "7915 ----\u003e T\n",
            "1248 ----\u003e ran\n",
            "7946 ----\u003e s\n",
            "7194 ----\u003e former \n",
            "13 ----\u003e is \n",
            "2799 ----\u003e awesome\n",
            "7877 ----\u003e .\n"
          ]
        }
      ],
      "source": [
        "for ts in tokenized_string:\n",
        "  print ('{} ----\u003e {}'.format(ts, tokenizer_en.decode([ts])))"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 0,
      "metadata": {
        "colab": {},
        "colab_type": "code",
        "id": "bcRp7VcQ5m6g"
      },
      "outputs": [],
      "source": [
        "BUFFER_SIZE = 20000\n",
        "BATCH_SIZE = 64"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "kGi4PoVakxdc"
      },
      "source": [
        "Add a start and end token to the input and target. "
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 0,
      "metadata": {
        "colab": {},
        "colab_type": "code",
        "id": "UZwnPr4R055s"
      },
      "outputs": [],
      "source": [
        "def encode(lang1, lang2):\n",
        "  lang1 = [tokenizer_pt.vocab_size] + tokenizer_pt.encode(\n",
        "      lang1.numpy()) + [tokenizer_pt.vocab_size+1]\n",
        "\n",
        "  lang2 = [tokenizer_en.vocab_size] + tokenizer_en.encode(\n",
        "      lang2.numpy()) + [tokenizer_en.vocab_size+1]\n",
        "  \n",
        "  return lang1, lang2"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "6JrGp5Gek6Ql"
      },
      "source": [
        "Note: To keep this example small and relatively fast, drop examples with a length of over 40 tokens."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 0,
      "metadata": {
        "colab": {},
        "colab_type": "code",
        "id": "2QEgbjntk6Yf"
      },
      "outputs": [],
      "source": [
        "MAX_LENGTH = 40"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 0,
      "metadata": {
        "colab": {},
        "colab_type": "code",
        "id": "c081xPGv1CPI"
      },
      "outputs": [],
      "source": [
        "def filter_max_length(x, y, max_length=MAX_LENGTH):\n",
        "  return tf.logical_and(tf.size(x) \u003c= max_length,\n",
        "                        tf.size(y) \u003c= max_length)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "Tx1sFbR-9fRs"
      },
      "source": [
        "Operations inside `.map()` run in graph mode and receive a graph tensor that do not have a numpy attribute. The `tokenizer` expects a string or Unicode symbol to encode it into integers. Hence, you need to run the encoding inside a `tf.py_function`, which receives an eager tensor having a numpy attribute that contains the string value."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 0,
      "metadata": {
        "colab": {},
        "colab_type": "code",
        "id": "Mah1cS-P70Iz"
      },
      "outputs": [],
      "source": [
        "def tf_encode(pt, en):\n",
        "  return tf.py_function(encode, [pt, en], [tf.int64, tf.int64])"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 0,
      "metadata": {
        "colab": {},
        "colab_type": "code",
        "id": "9mk9AZdZ5bcS"
      },
      "outputs": [],
      "source": [
        "train_dataset = train_examples.map(tf_encode)\n",
        "train_dataset = train_dataset.filter(filter_max_length)\n",
        "# cache the dataset to memory to get a speedup while reading from it.\n",
        "train_dataset = train_dataset.cache()\n",
        "train_dataset = train_dataset.shuffle(BUFFER_SIZE).padded_batch(\n",
        "    BATCH_SIZE, padded_shapes=([-1], [-1]))\n",
        "train_dataset = train_dataset.prefetch(tf.data.experimental.AUTOTUNE)\n",
        "\n",
        "\n",
        "val_dataset = val_examples.map(tf_encode)\n",
        "val_dataset = val_dataset.filter(filter_max_length).padded_batch(\n",
        "    BATCH_SIZE, padded_shapes=([-1], [-1]))"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 0,
      "metadata": {
        "colab": {
          "height": 305
        },
        "colab_type": "code",
        "id": "_fXvfYVfQr2n",
        "outputId": "98244f2a-a296-4ec2-f3a4-1db6f4c8460c"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "(\u003ctf.Tensor: id=311487, shape=(64, 40), dtype=int64, numpy=\n",
              " array([[8214, 1259,    5, ...,    0,    0,    0],\n",
              "        [8214,  299,   13, ...,    0,    0,    0],\n",
              "        [8214,   59,    8, ...,    0,    0,    0],\n",
              "        ..., \n",
              "        [8214,   95,    3, ...,    0,    0,    0],\n",
              "        [8214, 5157,    1, ...,    0,    0,    0],\n",
              "        [8214, 4479, 7990, ...,    0,    0,    0]])\u003e,\n",
              " \u003ctf.Tensor: id=311488, shape=(64, 40), dtype=int64, numpy=\n",
              " array([[8087,   18,   12, ...,    0,    0,    0],\n",
              "        [8087,  634,   30, ...,    0,    0,    0],\n",
              "        [8087,   16,   13, ...,    0,    0,    0],\n",
              "        ..., \n",
              "        [8087,   12,   20, ...,    0,    0,    0],\n",
              "        [8087,   17, 4981, ...,    0,    0,    0],\n",
              "        [8087,   12, 5453, ...,    0,    0,    0]])\u003e)"
            ]
          },
          "execution_count": 0,
          "metadata": {
            "tags": []
          },
          "output_type": "execute_result"
        }
      ],
      "source": [
        "pt_batch, en_batch = next(iter(val_dataset))\n",
        "pt_batch, en_batch"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "nBQuibYA4n0n"
      },
      "source": [
        "## Positional encoding\n",
        "\n",
        "Since this model doesn't contain any recurrence or convolution, positional encoding is added to give the model some information about the relative position of the words in the sentence. \n",
        "\n",
        "The positional encoding vector is added to the embedding vector. Embeddings represent a token in a d-dimensional space where tokens with similar meaning will be closer to each other. But the embeddings do not encode the relative position of words in a sentence. So after adding the positional encoding, words will be closer to each other based on the *similarity of their meaning and their position in the sentence*, in the d-dimensional space.\n",
        "\n",
        "See the notebook on [positional encoding](https://github.com/tensorflow/examples/blob/master/community/en/position_encoding.ipynb) to learn more about it. The formula for calculating the positional encoding is as follows:\n",
        "\n",
        "$$\\Large{PE_{(pos, 2i)} = sin(pos / 10000^{2i / d_{model}})} $$\n",
        "$$\\Large{PE_{(pos, 2i+1)} = cos(pos / 10000^{2i / d_{model}})} $$"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 0,
      "metadata": {
        "colab": {},
        "colab_type": "code",
        "id": "WhIOZjMNKujn"
      },
      "outputs": [],
      "source": [
        "def get_angles(pos, i, d_model):\n",
        "  angle_rates = 1 / np.power(10000, (2 * (i//2)) / np.float32(d_model))\n",
        "  return pos * angle_rates"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 0,
      "metadata": {
        "colab": {},
        "colab_type": "code",
        "id": "1Rz82wEs5biZ"
      },
      "outputs": [],
      "source": [
        "def positional_encoding(position, d_model):\n",
        "  angle_rads = get_angles(np.arange(position)[:, np.newaxis],\n",
        "                          np.arange(d_model)[np.newaxis, :],\n",
        "                          d_model)\n",
        "  \n",
        "  # apply sin to even indices in the array; 2i\n",
        "  sines = np.sin(angle_rads[:, 0::2])\n",
        "  \n",
        "  # apply cos to odd indices in the array; 2i+1\n",
        "  cosines = np.cos(angle_rads[:, 1::2])\n",
        "  \n",
        "  pos_encoding = np.concatenate([sines, cosines], axis=-1)\n",
        "  \n",
        "  pos_encoding = pos_encoding[np.newaxis, ...]\n",
        "    \n",
        "  return tf.cast(pos_encoding, dtype=tf.float32)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 0,
      "metadata": {
        "colab": {
          "height": 315
        },
        "colab_type": "code",
        "id": "1kLCla68EloE",
        "outputId": "234ee632-2612-401e-f445-e5d9d25e0819"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "(1, 50, 512)\n"
          ]
        },
        {
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYQAAAEYCAYAAABcGYHrAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzsnXd4VFX6xz/33plJJpn03iihGAIiYEVwVwQUKUFsyyKu\nHRssYmVdEWRtoKvrTyyLri4KiqCrEAVsqCgiWCDSpBoS0nsymXrvnd8fk5lkQiCFIsHzeZ7zzMyd\nk3vPHYb7zjnv/X5fyePxeBAIBALB7x75tx6AQCAQCE4OREAQCAQCASACgkAgEAgaEAFBIBAIBIAI\nCAKBQCBooNMHhNraWp5//nlqa2t/66EcM061cxLnc3Jzqp0PnJrndCI4oQHhoosuYvTo0Vx22WVM\nmDCB9evXA7BlyxbGjx/PqFGjuOmmm6isrGzzPmtra1mwYMEp9Q9/qp2TOJ+Tm1PtfKDzntO8efMY\nPnw4GRkZ7N27t8U+uq7zyCOPMHLkSC655BKWL19+zI5vOGZ7agOSJPH888/To0ePgO33338/8+bN\nY+DAgbz00ks8/fTTPP744ydyaAKBQPCbM3LkSK6//nomTZp02D4rV64kPz+fTz/9lMrKSiZMmMCQ\nIUNITk4+6uOf0BmCx+OhuQ5u69atBAUFMXDgQAAmTpzI6tWrT+SwBAKB4KRg0KBBJCQkHHKdbMrq\n1au5+uqrAYiOjmbEiBGsWbPmmBz/hM4QAO699148Hg9nnnkmM2bMoKioiJSUFP/7UVFRgHfKFx4e\nfqKHJxAIBCc1hYWFAbOBpKQkioqKjsm+T2hAePvtt0lISMDtdvPYY48xd+5cRo4ceUi/w0XH2tra\nQ9YEi4uLGTRoEIqiHJcx/xYoikJKSsopc07ifE5uTrXzgSOfk9Vqpbq6+pDt4eHhx+VHaEvXreN5\nvKPhhAaEhIQEAIxGI5MmTeKOO+7guuuuo6CgwN+nsrISSZJa/KAWLVrEggULArYNGjSIt99++/gO\n/ASTlJTE2rVrf+thHDPE+ZzcnGrnA0c+J4vFwi233MJPP/0UsH3q1KlMmzatTfuvrLESHWFpU1+T\nycTll19OTU1Nh4/XlOTkZAoLC+nXrx/AIassR8MJCwh2ux1N07BYvB/iRx99RGZmJn379sXpdPLT\nTz8xaNAgli5dyqWXXtriPq677jomTJgQsM33C8DpdCJby1mW5yFl2i3cEt6Phx79Kweu+jOb757H\ngJ4xBE29hfvW/JPMe7J5ZM6thN98A4oEySve4+q//gtddfPInFsZ9vVLvPTsV/QLDyLrtXtZLp3O\nc//5kqr9OcT0Oovx4wYyvHccZ+m5/Prvl9m8ei/bap1oHugaYqRfeiRdLzqNmD/+EU+vwdR5TGwr\nq2fjgWo2/1pBRaGVuvIK7NXFuOpr0Bw2PLoGgGwwgSxjMAWjBIVgNIdhCArBGBJGUEgQpmAjxiAF\nU5ARU7CCwSATbjYQEmQk1KQQGmQgNEghJMhAkCxhMiiYFJkgg4wsQZAiY5AljIqEQZIxKhKyDIok\nIQOyJCFJIEsg4b0RQJK8ySZJkvzbofF98Pb3bWv62rux8UXA9ib9myNLh3un2d8fodtjGcNZ2mcw\nV+/YQPaQy1gauZX0ZxYyZt46Rr45lyFdIri71+U8ee84vr54Ag99+DiDHtvE2WMu4sGcl3lz+Q4e\nfPfv/GlDOPu+3UBQWDSvPXwlUW88wuL/bMIsy1x549mETXuUh1bvYv3H31N7cBfmmGTCk3ty3vnd\nmTK4Kz3UAmo//R/7V21h184KfrW5sKkejDJEGRXCjArdwkxE944ipnciURldMXU/DSWxO2p4MnW6\ngSqnRonVRZ1TpdTmorDSTmmdgwqrC5vVjdPhwuXQUF0qbpeG6rSjOR14PDoe1YWmudHdbnTNDbqG\nrql4dA294Xvn0XTvo8f7Gl33f46+7+bhXncmUuKj+PK/D/LMM8+gaYHn0Z5f69ERFobdNI+Ckqoj\nHy8hii/+8wArVqw4quM1ZdSoUSxbtoyRI0dSVVXF559/zuLFizu0r+acsIBQXl7OX//6V3RdR9d1\nevTowcMPP4wkScyfP59Zs2bhcrlITU3lqaeeanEfR5pieQB0DavLg/1gIQeikqhzqlQdKKC4xkGl\nzU3wgQJwO8krqsTq0jDkF6JI4NJ08oqr0N0urC4NraaSygMFWCODwWnDKmscLKujvKgCe5SVKrsb\np6qD7kStKMOaX0hltQPN4yEq1ITD7ECrjgWnDTw6useD3a1TbXdTXOukrMpOTbkVW0UNTmsVqt0a\nEBAkWcEQZMYQ7MIY6sEYrGMMVQh26ASZVUxmA0HBGkFuAwaDjFPTsagenKoBt+5B9XjQALciE6SD\navCgezzIkoTHA0ZZQvdI6DJ4kFA8EnpDEFAkAgKC3PDaQ8Oj1HgRljn0gt7S9bnptrZa6x4LC97K\nAwc5EFlBxYGD5PWsxu0uxePxUFhlp+pAAQ6DjfzIOuxuncoDBeB2kFdSTVerE3dFqXeb00ZxjYm8\nkmqCHUZcmo5aVU7VgQJcioxWnY7H46G83kV+WS3VRRWEqiFEBNkor3fh1jygudHrqnAWF2PNL6XK\n6sKq6phkCdmkgFHBHhGEK8KFGmfAUx/h/e5obv/3x615cKg6Nrfm/V7b3ZTXuyitc1Jf58Jhc+G0\nq7idDc1hQ3PavRd91YWmutDdLnTV1bDN3SwgNDzqgY/Nn7f0ujOSlJR01PsoLKslr+TQpaemSLLS\nruM9+uijfPrpp1RUVHD99dcTFRVFdnY2U6ZMYfr06fTt25fx48eTk5PDxRdfjCRJ3HnnnaSmph71\n+cAJDAhpaWm8//77Lb43YMAAsrOzT9RQBAKB4OiRZf8F/0h92sNDDz3EQw89dMj2hQsXNtmlzJw5\nc9q137Zywu8yEggEglMB2WBENppa7dOZ6PTWFT6+G34ZQ1/axbV573Dxl4voev44bgvexZ9HdOfn\nVR8y75J07JqHjcGZeHSNyf1iWV9h4/Qr+/LqxgO462uISO3NxT1jyP10Oy7dQ3rPKOh1Ll/tKsVW\nUYBsMBGREM/pKRF0iQjCtX871b9WUebUcOkeTLJEtEkhJNZMaGIMhphEXAYzVrdOld1NRb13Wu9y\nqmguO3rDNL75FFySFX+TZQXZYEIxGJBkCUluWOeXQZIllMM1SUKWJZSGZSBZ8j4H7/OA1rCo03Q9\nvrUV/KZfnOZ5gYBzOUL+4Hgz+cKuDJ78F64+PR6Ahe/u5NKXNvLGjAsA+OOrD1C5P4fRQfkoEuxO\nOA9bRSH3j+hNzsrd9AsPQh0whvztudgqCkno1Yu+MUYK1u+nxq3TPdRI7Fl9yat18evBWhxVJQCY\noxKxRIbSK8FCtFnBU15AfUE5tnI7VlXHpXsXxBQJzIqMxSBhshgJCg/CFBaCEmpBDg7FYwjGYwjC\npXlwad7lIqeqY3d5H12qjqrqaJqOR/f4m643Lrj5loU8Dc27TW/6MfmXi1riVFgeOl7Ivv+frbTO\nhJghCAQCQQfw/WhrrU9nQgQEgUAg6ACS1HoOQZI61yKMCAgCgUDQASRZRmoladza+ycbnWu0R2Bd\nUR2b33+bZ697hZu+1XnrgQv5dMIDDHrtJTyaRvm8u7g0LZwHlv9MfN8haCuexa556HrdZNZ9m4cx\nNIKUzNNIqf+V3dvKMCsSqed3o1iOZPe+Shw15ZhCI4hKsJAZZyEx2IMjdx+1B2upVb3rrBaDTLRJ\nwRIfSlB8LHpoNFaXjtWlU+lwU2F14rK7Ud0aqsvuv/XPhz93oDTkD4ymJq8lFEVGUWRkg4xikFFk\nCZPvsUFr4MsheHMG3jyC7NcKSIddx/frCKTG53ILGoSmSM1EAMcqRXAsNAgApkUrWDtG4byN63j0\nHzdxdpSZjUuXkbn+Bf48rhfr4ocR2/tstv/9YS7pGsmDH+4gNC6N80Kq+b7KwaDBKXz2azVVB7Yh\nyQoD+ydi2LOe/b+UA5DWLZLg0wezpaiOyqI6XPU1KCYz5qh4ImJD6BEbSpQJ1JI86osrsFV6cwha\ngxLfJEuYFQmzInvzB6FGTOEhyKHhyKFh6EYzbg+4dA82t4ZD1XFo3hyCvSGf4MsfBD5qeDQt4BbS\n5reVtkRLt5wKjkCTXN/hGmLJSCAQCE59pDbcdtrZZggiIAgEAkEHkA2mNtx2euT3TzZEQBAIBIIO\ncCrOEDrXaI/A9EfGMeiKSbg9Hpa/+Ca91jzFigM1zMnR6T8mi+znvuYPT1zF9s+/ZuSofvz4r1X0\nCQuiqvdwCrf9QEzPQQw/Jw37uvfZbXXRLcRE0rDz+LGwjvKCanTVRUhMMqd1jSQt3IhSlUfV7nwq\nK7zrwgChikxItJmQpGiUmCT0kCisLq8GodLqotrqwuXUUO1WdLfXTqAlDYJXe2D0PjeYvDoERW7Q\nIXjzApLUJH9gUPy5A982n/bAtx7v0yAokoQiNz5KAVYUra/dt2V5v616g+MtSxh+w9M8c9ZNnP33\nT7mp5AMmrXyEkJhklt65mL4vvMhd/97IVROHsnLFHobMHs+mTzeTfu5gqt9egFXV6XPtMF77Nhd3\nfQ0hMcmM759E9Vcfs6/eTbRJIWlQEu6kvvxwoIq60iJ01YUpNIKwaDNxsSF0izSj1BbjKjpIXZGV\nSpeGQ/egeZpqELzN2JA/MIWHIoeG4zGFgjEYh+rB1WBb4WxodpeGq+FR8+cRaKJH0AKaj7bkDwRt\nR5JazyFIksghCAQCwSmPLMutCs/kTjZDEAFBIBAIOkIbhGniLiOBQCD4HSCUygKBQCAATk2lcuca\n7RF4s/tVfH1LD+7592SCLFG8cve7XJIQysKFq/nvLeeQU+OgfvQMrCW5PDSiJ1/+XMqQoam8va2E\n+rJ8up+exqRBKexbsYFKl0ZGYiimgcP4ck85dYV7kWQFS0IaA7tGEumpR8//heq9JRQ7NOxao7Fd\naEIIoYnRGGIT0cyR1Do1ym0uSmudfr96zWVHc9kPMRVrKkpr3rxiNK84TVYkvzDNcBhzu6bGdn6h\nmtxUoCYFCMCaG9sdSRzWEWO73wJLQncA9nzxPs9d8yLPqYO4+56r+L7KwSObXez5YgXzLkkn3+6G\nKx+gYu9P3D6uDzmvfkOa2YhpxF/YvrkIxWQmttcAhnaJIH/tVsqcKmlmI4nnZlLgVMjJrcReVQxA\nUEQskXGh9EkKJ8liQqktpi6vhPqSemrcOvaGQjQmWSJYlrAYZILCTASHB2EKC8VoCUUOCcNjMqMb\nzQ3Gdh5sbh2n2ihIc6kaqqqjax50VUdTdb+xXdOEckvGdk3fOxItJZpF8rkR3w0frbX2kpuby8SJ\nExk1ahQTJ04kLy/vkD6VlZXceuutZGVlMXr0aObOnYvezLSwQ+d01HsQCASC3yGSInt/wB2xtf8S\nO3v2bCZPnsyaNWuYNGkSs2bNOqTPyy+/TI8ePVi5ciXZ2dls27aNTz755KjPSQQEgUAg6ADH47bT\nyspKdu7cyZgxYwAYO3YsO3bsoKoqsFSnJEnU19fj8XhwOByoquqvWX80iByCQCAQdID2CNOKiopa\nrKncvCRwUVERCQkJjbXKZZn4+HiKi4uJiory97vjjjuYNm0aQ4cOxW63M3nyZAYOHHjU53TKzBCe\nf+p1PjxjLB/0u5nZsyZT6HAzZvksqnO30WXdy5wdFcw92TuJ6NKHlJ0fUehQ6XvbeBZ/thfFZOba\nC7qTabaxf10eJlmiy5BUqiN78MOuMuxVJRjMFmISwzg9PgxD+X5vcZwDNVS5mxnbJYQSkhgDYbHU\nNeQPym1uKuudOB1u3E41wNjOtybbtCCO77nSII1XDHKD0VyjME2RJYIMsl+IFpBLaGJsB/jzB7J0\naG5AJtC0runzY2ls11axWluN7drC9oWTuHvTQobdchP1ms7jj73JzKRCrsiI4dVXP8EclUD5vLs4\nL9rM41/8isFsYXK/WL7ZX8X5p0WzxR5G2a4fiUjtTXrfeKIrfiF/YyGaB9LjQwgdMJhtpfWUF9Th\nrKtCNpgIiUmhW0IYvRIsxJgV3Af3YS0oO6Q4jtfYTsZsUjCFeovjBEVakELDkULD8RiCUJGbFMbR\nAoztbC4NTfPmDvQmhXGaG9s1pcVt2qlXL/mE0Q5zu2uuuYbhw4cHtEWLFnX40GvWrCEjI4P169ez\nbt06Nm3adEyWjMQMQSAQCDqALDf+6DpSH4AlS5a0OENoTlJSEiUlJXg8HiRJQtd1SktLSUxMDOi3\nePFiHn/8cQAsFgvDhw9n48aNXHzxxUdzSqfODEEgEAhOJL4Z+xFbw4w3KSmJ1NTUgNZSQIiOjiYj\nI4Ps7GwAsrOzyczMDFguAkhNTeXrr78GwOVysWHDBnr16nXU5yQCgkAgEHQAX32S1lp7mTNnDosX\nL2bUqFG89dZbzJ07F4ApU6awfft2AB588EF++OEHsrKyuPzyy0lPT+fqq68+6nMSS0YCgUDQASQZ\npFaWjDqiS0tPT2fZsmWHbF+4cKH/eVpaGq+99lr7d94Kp8wMIbHfEL4os3HXQ4u4zfkNN17Vh3dC\nh9LjwstYNXURo+8bzqfvr+eMkYP5+YlXSDMb4eIp5P6UQ1S3fozpHYv2zbvk1DhJDjaQNuIsNhfX\nU5JXjeqwEhKTTLcuEfSICsa192eqdh6gusga4HQaER1MaGIkhrgUtNAYrG6dCpuLcquTCqsLl13F\n7bChOe2HOFFCk4ppstxYLU1WAqukGWRkpTGZrMgyShPnU58IrWk7ktNpwPFbyeeeTE6nbRnLVz3P\nZvjHEqtHwL0v/BnVUc+aS/7KRcvnU7k/h/OvHE32c19z8YwLWbpiB13OHoa24lmKHSqn3zCUV747\ngK2ikJTM05h0bhdsX69ka40Ti0Em5ewk9O4D2XSgipqSclSHtcHp1ELflHB6RIUQVF+GuygXa1Ed\nNQ6Vek33O52aZAmLQfImk8ODMIWbvU6nlkgICsVjCsGpeXDpngCXU19zNYjRNK1BnObx4PEcRpR2\nmCSz4Chpw5JRm76oJxFihiAQCAQdoLna/3B9OhMiIAgEAkEH8M8CWunTmRABQSAQCDrAqRgQTpkc\nwsezhvHAwxfjttWy+MrH6fn6//jbPz9m4bQhfFZaT8Rd/6Ryfw7PXdGfz77K48JBiby9rZTag7vp\nPvA0Ekq3kPv+Z97142gzIYNH8/nuMqrzdwMQnpTOuT1iiDO4sO7eReXuQgrsKnZNbzS2iw/FkhKH\nEpeCHhJFrVOjrMHYzlbvwml3e6ulqS40d2C1NJ+xnWwwIhtMAdXS/GK0BmM7nzDNVyGtsXJaY7U0\nf97An1Pwvm5ubNe8WprP2O5wM92WjO1O1q/8d5V2vn1jEa+eexOrz7qdP0+dTPbBWhZae5B27hje\nvGYAOTUOYqY/QdHmz7jxskx+/NcqEoMNRF5xM19vyEM2mBg6KIUR6dHkrt5ESYOxXdLgTIoJZ8Oe\ncurLvOZjXmO7EHrGhJIWEYRSU0hdXom/WprP2E6RJIJlb6U0k8UnSgvDGBaCHOo1tvMEGNtp2Nze\n3IHNpeFSvTkEXWsUpDUa3GktNji8sV1b8wsiDxGIokhew8kjNeVk/d/RMmKGIBAIBB1Bklp39BU5\nBIFAIDj1aY9SubMgAoJAIBB0gOOlQ/gtEQFBIBAIOoDPuqK1Pp2JTha/Ds93wy/jx6se4d6/30hO\njZNLX9pI6Y71nLVjKWdEBHNX9i+Ep/amd95acm1uBs4Yz8I1u5FkhckXplP2wVL2rNmPIkH3YV2p\njc/k620l2MoLMYZGEJ0UxoCkcIxle6naeYDKPZWUuzQ0D5gVibgghbBkC6EpcUiRCdSpEiVWF8XV\nDsrqHDhsjU6nmsvRLqdTXzLZl1w2GOQGp1OlRadT31RWkRqdTpUW7pn2J5L9CWeavd82p9OWaNrn\nt3A6BZjz5XwGT/4LuTY3Ux96k5fPUhnfNYJ/PLuax24/D9eC+xgUGcy870qQDSbuPCeVL38u5Q+9\notkhJVO8/UfCU3vz50EpJNbs4cBX+bh0D6fFhRB+7gX8XFJP6cFaHDXljU6nSWH0jAkhPsTgdTpt\nUi3N53RqVryV0ixBBoKjgjFHBWMKD0EOi0IODcdjNOOWDDhUHYfb63ba3OlUdWteYZqqo/sdT4+d\n06lIILdOcwHo4VpnQswQBAKBoAOciredioAgEAgEHcBnJ9Nan86ECAgCgUDQASRav+1UOmlVOi3z\nm4SvBQsWkJGRwd69ewHYsmUL48ePZ9SoUdx0001UVla2e5/riuq4/u6FPCB9y5SJmWxcuoxewybw\n0U0vcvmDI1n5zleceekF5DzyvNfYbvRU9m78gej0M5jQJ45d7/3IT9UO0sxGul56Lj8U1VP0a6Xf\n2K5X9yh6R5tx7dlC+a4yKgobje3CDcohxnY1To0Km4vSOieltc52GNsp7TC2kzA1ey1LEkZZPsTY\nTj6MsZ1fXNZC/iBgfJ3M2A5g5LcxrL0UZr58Dc6aclYNvYGLVz1H+e7vuZLtvPfkZ4y7fzivvbOF\n7oNHov9vPoUOlTOm/JHn1u2nviyftH6ZDIyC+i/+R061A4tBJvW8ZDw9z2H9/gqqCor9xnbhsRH0\nT4skIcTgNbYr2EftwRoq7e5Wje2CIsO8xnbm8ABjO2+1NN0vSjvWxnYiV9BxvHcZtd7aS25uLhMn\nTmTUqFFMnDiRvLy8FvutWrWKcePGMW7cOLKysjp03WzOCQ8IO3bsICcnh+TkZP+2+++/nzlz5rBm\nzRrOOussnn766RM9LIFAIGgXvps3WmvtZfbs2UyePJk1a9YwadIkZs2adUifrVu38uKLL7Jo0SKy\ns7N56623sFgsR39OR72HduByuZg7dy5z5szxb9u6dStBQUH+AtETJ05k9erVJ3JYAoFA0G58OoQj\nt/bts7Kykp07dzJmzBgAxo4dy44dO6iqqgrot2jRIm688Uaio6MBbxlNk8l01Od0QnMI//d//8f4\n8eNJSUnxbysqKgp47SsVV1tbe0iJudraWmprawO2KYpCUlLScRy1QCA41SgqKmqxxnFLZS0PSzus\nK9p6vKKiIhISEvz7lWWZ+Ph4iouLA8po7tu3j9TUVCZPnozNZmPkyJHcfvvtbR/7YThhAWHLli1s\n3bqVe++917/N4/G02Pdw2xctWsSCBQsCtqWkpLB27dpjN1CBQHDKc80111BQUBCwberUqUybNq3N\n+5ClNlhXNFzYj8XxmqKqKrt37+a///0vTqeTm2++meTkZMaPH9+h/fk4YQFh06ZN/PrrrwwfPhyP\nx0NJSQk333wz1157bcAHVVlZiSRJLUbq6667jgkTJgRsUxQFgLv+kcWz/63n5cufZHLBZkL+9CxL\n7/sDLzxbz7Db51H59lReePlG3rrnAJcMTeO1LUXUHtzNuZOuJS7/O97ZWUGxQ+Wy9CiCzx/Hqi0l\nVOf9giQrRKT04PxescTLNqq3bad6f3WA02lckNfpNKxLAobELrhDY6gpd1JsdVJa68BR726706nx\nUKdTr2uiV5ym+J1NFUyKHOB62tTpVJECnU4VuWWnU0k69k6nJ4s6c8PiN3hm9jeYP/2Y6X9z8OHo\nd8kpSib9D+NZd+1DbKt1cuNd/6Ro6DT+9cLf2TTlH6SZjYRfdQffPJqDYjJz8Xld4IcP2f/hRgod\nbvqEBZH6xzMo0EL5bs9uv9NpcFQC0QmhnBZnIUJyolTlU/trUYPTqe53OjXJkre6nlEhKNxEcKQ3\noWyKCAtwOnW4PDjVRqdTq0PF7vY6nTpdmjeZ3ESY1qLTqda606mP1kRpp1ryecmSJS3+Ym8PBoOM\noZXbSn3vt/V4SUlJlJSU4PF4kCQJXdcpLS0lMTExoF9KSgqXXHIJBoMBg8HA8OHD2bp1a+cJCFOm\nTGHKlCn+1xdddBGvvPKKv37oTz/9xKBBg1i6dCmXXnppi/to95ROIBAIWuBYLDP7bOhb69Oe40VH\nR5ORkUF2djZZWVlkZ2eTmZkZsFwE3tzCunXrGD9+PG63mw0bNjBq1KiOnUgTfjMdgiRJ/ig4f/58\nZs2ahcvlIjU1laeeeuq3GpZAIBC0CUVqPSAoHZgtz5kzh5kzZ/Liiy8SERHB/PnzAe+P6unTp9O3\nb1/GjBnDtm3bGD16NIqiMHToUK666qoOnUdTfrOA8Pnnn/ufDxgwgOzs7N9qKAKBQNBulDbMEFp7\nvyV8qybNWbhwof+5JEnMnDmTmTNntnv/R6Jz6aqPwBtdr+TfT99KocPNiCe/4u57riL1gycYFhfC\ntUu2ENNzEKnfL6bQoTLgbzfwavZODMEWbh/Ri8Klb7Gv3oVJluhxcQ/KInvy7dZibBWFmEIjiEuN\n4KzkCJTiXVRs/5XCWiflLhXNAxaDTFyQQnhqGKHJ8RART7VLp7TeSXG1g7p6Fw5bQ7W0BmM7XXUd\n1thONpgOMbaTJAnZIPunqEFNhGjNje2MirefX5jWsL05MoGmdS19bztqbBdwnOZ/f9h+bdtvew5/\n09+nA/C3B15glvknrjkvhWf+uZzXZgzlvW2ljIgP5a7sXwgKi+amjBA+21XBhYMS+dYaRtHWDUR1\n68fkQakcXLGKvd8cRPNARmo4YecP5/vCOooPVOOoKUcxmQmN68JpaZH0jA7BUJWPO283tblFWAut\n1Lg1v7GdV5QmYzYpmKOCCYoKIyjSghwWiRwWhccUiksy4NJ0f/6gvqFamt2lYnNpaFpD7kBvrJh2\naM6glZyAdmrlBH4LZLkxKByuyZ3sCiusKwQCgaADGBp+kLXWpzMhAoJAIBB0ALkNOQRhfy0QCAS/\nA0wGBZNBabVPZ0IEBIFAIOgA7bnttLPQyVIeh+f5p17ngo8e457Hx7F91XJmJhXy+v3/I2vhLXz7\n7iqunvRHvp35OmdEBFM24HJyN31LfN8hTOgTy45lm3HpHnpbTKSNG866AzUU7S9Fc9mxJHTj9J4x\n9IgKxrl9I+W/VFDs0PxOp1FGhYj4UMK7xKMkdEELS6DGqVFa73U6tde5cNpVVLsV1W5FU10tOp36\nhGhyk+d4RpboAAAgAElEQVRNnU5lWQpwOjX5q6Y1bjMekkz2JnUbnU8bnU79bqcECtG8QrXA5PGx\ndDo9kTxZs5y7Ny0kNC6Nl7Pmcu7q97FVFDLgx9dJMxsZveA6Vr7zFX0uGkHFS49Q6dIYOGM88z7d\njaOmjPRBGfSWK9i7ejc5NQ6iTQpd/9AVtfs5fLG7jJrCAjSXHVNoBJFxofRPiyAx1IBWuBfHgX3U\nHqyl3KkGOJ2aFdmbVI4KJjgqmKBIC0FRYchhURAchh5kwaF6cGo+UVqj06nP7bSp06n3UT/E6RQ4\nxFW3+feupQS0oO34bjs9YhNLRgKBQHDq47vLqLU+nQkREAQCgaADKFLrMwAxQxAIBILfAcdLmPZb\n0skmNIcnsd8Q5s9axY9jHyTt3DGsueSv5Nrc5Ay8DntVCfMuSWfVznJGTu7PY2v3Yaso5Nyh3ZHW\nLWFjfi1pZiP9M2KQzxnHB1sKqMnbgWwwEZnahQt6xhJpL6Hy512U/1pNlVvDrnkwK15jO0uyhbAu\nCRiTu+E0hVFhc1NU66Co2o7D5sZR70J1eI3t9MMY20myjGw0NeYPDIaG/EGgsZ03Z6Acsn5pahCk\n+YztmucNlCbGdn4zu6a5A458i9yRjO3kgByEdEi/psc4kTx4y2KGfyyR/exfKHS4ueS/v3DuxKtZ\nNuU/TJx6PrkX3Ebl/hyenjyIdc9+waDIYBg9lZ+/3klwRBzXXpiOa+0SNhfUUebU6BNmImXEeeyu\n1dmyuxxrSS4AITHJxKeEkxlnweKsxJ27k5q9BdTm11Hp8n5XoKmxnUxwVDDBUaEER4ZhjIxEtkTi\nCQrFYzTjVHUcqo7VFWhsZ3dpqKqO6m5qaNdEmNaKsR3graR2BFHa78HY7lghtyGHIG47FQgEgt8B\nvhs7WuvTmRABQSAQCDrAqbhkJAKCQCAQdIDj5Xb6W9K55jMCgUBwknC8zO1yc3OZOHEio0aNYuLE\nieTl5R227/79+xkwYIDfIvtoOWUCwsezhnFetJnr/raElbNHkn2wlmsu6sZNz62n17DxlM+7C83j\nofsDs1i1ajuhcWncP7w3e15/j0KHyplJFnpdNpC9ajhbt5ViryrBHJVAcvcozk4Jx7P/J8pyDpBn\nU/3VryKMCnEhRiK6RBKcloYWnkiVQ6OozsnBSjv1dV6XU5fdjtthRXM5Dk0oy4rX3dTQpFKa0YSs\neJ1Om4rTTAYZQ4MozdRMlOZLIMv+543T1UaxWuPn5U8MS4cXlclSs8Tzcfq1czycTgEmnZ/Gt28s\nIvKJW7j70TFsWPIWa247h+8q7cTOfpkbFm4kpucgzrdtYV25jT9elclrW4oo3/098ZnnMiEjlt3v\nrCPf7sYkS/TsH0/QOZfwVW4lpXnVOOsqMQRbCE9MZVDXKLpHBmOoPEDNvgJqckuoLrdRq+p+p1Of\nKC0kPMjrdBoZRlB0BHJYFHJ4NHpQKA7Ng0PzUOdscDptcDi1u1ScLg3VrXkTylpj8+ia/2YFvUli\nGURC+HjSqiitDUtKLTF79mwmT57MmjVrmDRpErNmzWqxn67rzJ49mxEjRhztqfg5ZQKCQCAQnEiO\nR0CorKxk586djBkzBvBWRtuxYwdVVVWH9F24cCEXXXQR3bp1OxanA4iAIBAIBB3CqMiHzNabN6PS\nvktsUVERCQkJ/tm4LMvEx8dTXFwc0O+XX35h/fr1XH/99cfqdACRVBYIBIIO0R5zu6KiIrRm+o+O\n1ohXVZWHH36YJ5544pgv454yAeG74ZdxxZb3ueGKF4h4+R7Gd41g0FuL2DvqH3y09HGyz76X0b2i\nWW2Np2TrOvqOuowB5PPmJ79iViR6Z/UhfvxVLNpZQsmeXXh0jfDU07iobwLdwhRsW3+kfFeFv1Ka\nSZaINXkrpYV3T8KY1A0tLJ7qGo0iq5OiGjt2q9NbKc1uRXPa0VoQpclGk//RVynNK0xrMLUzNOYS\n/L88FDmgaprJIAeK0fwmd4euzzc3tvMhS1KAsV1ANbUWPu/OcO9E5PKPGPzWT/zrvuuZsOt7on98\nlz03XsFl6VFMfiuHrauzueX+W9h8/11YDDK97r+fP722E4+uMXRoN6Jz1/PxN/loHugWYiR9VH8q\nwtP5eOvPVOf9gkfXMEclEJsSzhmpESSFyLh+2E7NvgKqc2soczaaIDYVpfmM7YJjwpEjYlAiYtCD\nQvGYvDkEp+rxV0uzOlSszqaV0jz+R90nTGswTGwqSoNA87qmpnfN3zsSIgdxeNpz2+k111xDQUFB\nwHtTp05l2rRpAduSkpIoKSnx15vXdZ3S0lISExP9fcrKysjPz2fKlCl4PB7q6uoAsFqtzJ0796jO\n6ZQJCAKBQHAiaY+X0ZIlS1qcITQnOjqajIwMsrOzycrKIjs7m8zMTKKiovx9kpKS2LBhg//1ggUL\nsNls3H///UdzOoAICAKBQNAhWrN78fUB70W8rcyZM4eZM2fy4osvEhER4b+ldMqUKUyfPp2+fft2\ncMStIwKCQCAQdACfb1hrfdpLeno6y5YtO2T7woULW+w/derU9h/kMIiAIBAIBB1AlqVWK6KJimm/\nEeuK6vjjGwXceO/NvPjkWi5e9Rw3flxKeGpv/lD6BTk1DobOu45HluUgyQq3j+tDyaIXyalxcEZE\nMKlXXkZ12jlkb8zHWpyLMTSCxG5xDO0WjalwKyU//EJBqY0at44igcUgkxhsILJbBOHdkpBiU6lR\nZYrqnBRU2qmoceCod+OurwkQpfmSdD5RmrdCmrGxYpovoexzOFW8jqcGf4U0JSCZ7E0oe7+YRkXy\nJ5d91Zp8TqeKjN/ptKngTJbaliD23wbXrPPROp0eTzfIC274F2svhaye0Uye+RYvzLqM15bvZMS7\nT/Dl0g+RFIV5l6Tz4RcHGNU3jh9Np5G76VuiuvVj2gXpFC59m221TpKDDfRPthAzfBTfFdTx6+4K\n7FUlyAYTlsTu9OoSSd94C8aK/dTv3UPV/mpqD/qcTr1JZbMiYTHIRAQbGhLKEQTHRCBHxCBHxOAx\nheKWTdjdujeZ7FIbnU5dqtfp1KWjur0V03RVR9c9aKrqdzZtqTqaR9cDPpP2Op0KDo9BljG20gyd\nrEKOmCEIBAJBBzheS0a/JSIgCAQCQQeQpTYsGXUyczsREAQCgaADSFLrBXCOl//X8aJzLXAdgbv+\nkcX37yzmX93yCVVknixKZuVr/+OGm0fz1c3zGBQZTOmQG9n91ZcknjGMyf1i+fn177BrHgZc2AV1\nUBaf7q8i/5cCVIeV8KQeDMqMp198CI6cbyjJKSbP5sale7AYZKKMCtEJoUR0S8CY2hMtIokqp0ZB\nnYODVTbsdS6cDjeqw+oVpqkudNUVMGa/mZ3B5De4M5iMAYZ2ikFGbiaRD2pmbGeUJYyyjCxJGBWf\nSK1RqOZrzUVpgcZ1jaK0pttOFjoyluCIOJ45ZwoX/vQFtYX7GJnzCsnBRt7QMnHUlNN/9FjK591F\nsUPlvEeu4W8rt2OrKKTXef05w1DGjmVbqHHrnBkbQo+Le6L2uZAPtxVTcSAX1WElOCKW2JRozk2P\nJi3MiJa3k+o9+dQerKPM5qZW1dA83mUDcxNRWkiMmeCYcAyR0ShhkRAchh4cjl314NQ81DhVrC5v\n/sDm0rC5NFw+YZqmo2uehkc9QJDW1NjO27z5g+a5gZZEaSJ/0H58S0attc6EmCEIBAJBB5DbMEMQ\nS0YCgUDwO6A9XkadBREQBAKBoAMYZVBbueAbO9mifCcb7uF5o+uV9M/6E6+NuIdpL/yZZ/65HNVl\n57Gzg/lgVwXj7h/O1Pe2Yq8qJuvS09BWPMu6g7X0tpjoPWkkn/1azRsbDlCduw3FZCa+RzoXZ8QT\nUfMrpZu2UvRrNVVu7zprlFEhMVghKj2SiJ4pGJLTsRksFNe5OFhlp6jagc3qwllXi9vu1SDoLRjb\nKUYTkiw3GtoFmf35A4NRxmBUMBgbi+MostfYztTU2E6Rvb9UGtYr/QZ3zTQI3hyB98vbXIPQfFob\noFNoth0aNQVHq0E43mx99ToAznl0PZNn3MiCKW9yyzNX8vCzn5IxMos3bj6H7Oe+ZkR8KNV/uIlt\nX/xIaFwad406jboP/sOmwjoijDI9LkkndewItpQ5+Wl7CdaSXCRZwZLQnbSukQxMCie0vgTH7m1U\n7SmjrMZBmVPDrjUWxgk3yESbFMyxZsyxYYTERyGHx4AlBj04DN0Ugl3VqXfp1DnVAGM7e0NhHNWt\nobq8OgStQYegu12B5nZay3oEvYnpXVsReYUjIyEdkqc7JG/XKWwgGxEzBIFAIOgA7XE77SyIgCAQ\nCAQdQD5C+dmmfToTIiAIBAJBB/AtzbbWpzNxQgPCnXfeSUFBAZIkERoaykMPPURGRga5ubnMnDmT\n6upqIiMjmT9/Pl26dDmRQxMIBIJ2cbxuO23L9fDFF19k1apVGAwGFEVhxowZDB06tN3HOmS8R72H\ndjBv3jw++OAD3n//fW644QYefPBBAGbPns3kyZNZs2YNkyZNYtasWe3e9/NPvc63009nZ52T9YPv\nxFZRyNBJV5Jz4xSSg41E3PVPNmR/RWzvs/nbRen8+K9VVLo0zj0jAcPwv7DouwPs/rkYR00ZlsRu\nZGTEcnZyGOq29RR9n8uv9W7smsdvapcYF0JE91iCu/ZAjUim3K5RUOvgQIWN+lonjnoXqsOK5rL7\nk34+fII036PiF6cZGhLKSoAozWxS/GI0X6U0X+U0WfKK0gwNzxWJJhXTmjQCzemaf0+P9EVoTW3Z\nUTVmW/+zdPRH1neZ53H3poXs/PhdXjytlCq3xp5R91G6Yz3/vfN8kj9/jpwaBxf+fRQPrdlN5f4c\n0s8dzNiuwWz97zoKHSqDIoPpmnUR8jljWbGtmOK9B3HWVRIUFk1USiJDesWSHhmEdHAHlb8coOrX\naoodGlVuDZfuaRClSUQYZUKizYTEhmCOi8IYGYkSFYceHIYnOAybW8eu6lhdKna3htWpUucztnOo\nqO4GQ7sGUZrHE1gprakoDbymdi0lhdu6TdA6ity21l7acj0844wzeO+99/jggw947LHHmDFjBi6X\nq4W9tY8TGhAsFov/eV1dHbIsU1lZyc6dOxkzZgwAY8eOZceOHVRVVZ3IoQkEAkG7MEhtcDuV2neJ\nbev1cMiQIQQFBQGQkZEBcEyumSc8h/DQQw+xfv16AF599VWKiopISEhotFeWZeLj4ykuLg4oGycQ\nCAQnE8djyagj18P333+ftLQ0EhIS2nWsljjhAeHRRx8FYOXKlcybN4/p06fj8Xja9Le1tbXU1tYG\nbFMUpV3l6QQCgaCoqKjFGsct1Tk+HIoMSiuXLt+S0bE4Xkts2rSJ559/ntdff/2o9uPjN7vLKCsr\ni1mzZpGUlERJSQkejwdJktB1ndLSUhITEw/5m0WLFrFgwYKAbSkpKaxdu5bEfkP49IxRzLjvQgY/\nsoJzJ17Nyuv6M/Ov+7nztrO556Nd1B7czZUzbidywxK+/LmUNLOR028axoYKmZ9/LKR890/IBhPx\nPTIYf0YyCa4SSjZ8T/GOcspdjaK0pBgzUd0jierdBWOX3tSHxFBaaiOv2s7BShu2WifOeiuu+ho0\nlwPVaQ8wtpNkBUnxFsZRgsxegzujCVnx5Q+kxgI5fiM7BZMiBxTG8ZrZNYrSjP6cwqGiNMBvbucX\nqIHf9M77frPiOU0+5yOJ0mihn/9cj/QlaANHc5PGuhIrT30sMeS661l80a3cec8fuejJL+h6/jhO\n+/51Vk5/i0GRwYTc8hgf37yE4Ig4poztg/rRi3y7rQyLQSbjwq4YBl/GL85QvszZSW3BbgAsCd1I\n7hbJmSkRRLqrcO7eTNWuIsrK7VS5Nayq11jOJEuEGxSiTQohsWZC48Mwx0SgRMUjh0WjBoehB4Vh\nt2vY3To1TpU6l0aNze3PI/hFaW7dXxxHU/VmRnbaYQvltCRKay1vcKrnFa655hoKCgoCtk2dOpVp\n06a1eR+S1IaaylL7jtee6+HmzZt54IEHeOmll+jatWubx30kTlhAsNls1NbW+k9s7dq1REZGEh0d\nTZ8+fcjOziYrK4vs7GwyMzNbnB5dd911TJgwIWCboignZPwCgeDUYcmSJS3+Ym8PTd2Dj9SnPceL\njo4mIyOj1evhzz//zN13381zzz3nzyEcC9odECoqKrDZbAHb0tLSWv07u93O9OnTsdvtyLJMZGQk\nL7/8MgBz5sxh5syZvPjii0RERDBv3rwW93EsplgCgUBwLJaZZaRW78rxzcbbc7zm18P58+cDMGXK\nFKZPn07fvn2ZO3cuTqeT2bNn+2cT8+fPp1evXh09HaAdAWHdunX8/e9/p7y8PGDNX5Ikdu7c2erf\nx8TE8M4777T4Xnp6OsuWLWvrUAQCgeA3pz0zhPZwuOvhwoUL/c/ffffd9u+4DbQ5IMydO5c77riD\nCRMmEBwcfFwGIxAIBJ0FWWpdZ9DZrCvafJNsbW0tEydOPGmDwcezhrGqoJaS25+hfPf3rLntHHbf\n/CcijAppTyzk/be/JLJbP54Y04efnlhMoUPlgtPjMI+9mX+v/5WyXT9iqyjEktiNXplxDOkSib5t\nHYUb97HX6saq6pgViRSzgaj0SKJ6xWHu0QstKo1ym0pejYP9ZfXUVjuw1Tlx19egueyoTnuLojSl\nSaU0xWTGYArCYPImlA1GBYNJwWBUDqmQ5m+K3JBQ9tZ1bU2UJkmBorRWPVh84/2NRWlHw9xP/sG3\nbyzis8vCyalxYLvrOfI2fMi/Zwzlo1tf5YsyG2PvGcZDn+yldMd6up/3R647PZYtL3xMrs3NGRFB\n9LpqGLvVSN7fVkzB7kIcNWUEhUUTnZbGH06Lo09sCHLBDip+3kfFniqKHSpWVfeL0iwGb6U0S1Qw\nIbFmzHFRBMXHokTFo5sj0IPCsKke6lWdmgZ301qHmzqHitXhxunS/KI0TdUbq6apLr/T6eFEaW1J\nDJ/qyePjiXfJqPXWmWhzQLjiiit47733judYBAKBoNPgWzJqrXUm2rxklJOTw5tvvskrr7xCbGxs\nwHtLliw55gMTCASCk5nftdvpVVddxVVXXXU8xyIQCASdBonWdTadLB60PSA0v///ZOO74Zdx331/\nZOjf/sd5k65hz41XsOjdX7j15kHc9fEBKvfnMGH6bST8+A6LNhWSZjYy4LYRfFcXyvcbD2KrKEQ2\nmIjrmckVg1JJVsso+fpbinNKKXGqAMSaDCTFmIk5LZ7ojG6YumVQHxpHUamN3EobB8rrDxGleXSt\nXaI0g1EJEKWZTYpflNY0l+Cz3jXKDa0VUZpPcNaSKE3mUFFaUzqjKA1g1JY0Bk/+C2+e+Wem3/MH\nRs79nC6DxzJ4x9vMKK1nUGQwkfc8y/IpSwkKi+aOCX3Rs/+PrzcXY1Yk+l/UDeOFE3lvSxGrvz9I\nde42wCtKS+wWyeCuUcSoVTh3bKJiZwGlpfWUu7ymdtAoSosLUgiND8WSFEFIfJRXlBYRi2qOwKZ6\nvMZ2hxGluZwquqp7RWla+0RpEFgprTHH8PsWpR0rfP9nWuvTmWiXDuG9995jxYoVlJSUkJCQwPjx\n47niiiuO19gEAoHgpEVpwxRB6VzxoO0B4aWXXuKDDz7gxhtvJDk5mcLCQl599VVKS0u5/fbbj+cY\nBQKB4KRDamL7cqQ+nYk2B4Tly5fz5ptvkpKS4t82dOhQJk+eLAKCQCD43SFL4Pm9JpXtdjvR0dEB\n2yIjI3E4HMd8UAKBQHCyI9EGpfIJGcmxo806hAsuuIB7772X/fv343A42LdvHzNnzjwmZduOBeuK\n6th789OU7/6eT27pz2vLdxJukEl75k3effMTYnoO4pnxffl+9msUOlQuHJRI8GV38K8v91K683tk\ng4mw5B70PyORYd0i0TZ/Sv7Xu9lV58Kq6lgMMilmAzE9o4jpm05Ir9NQo7tSZlPJrbIfVpTWPKHc\nmihNUeQ2i9KMcttFaU0dT38PojSA9Yv+y9pLYVutk+q/PseBb7N5e+Yw3r3xJc6OCmbCw5dyz0e7\nKNm2jp4XDOf6zAh+eCabXJubs6PM9J50MTvdEazamM/Bnfk4asoIjogjtnt3RvRNIDMuBDl/G2Vb\n9lC+q4ICu+p3OfWJ0qJNXlFaaEJooygtJrFRlObWWxWlqW6t0eW0HaI0XSSPjysS3v8rR2qnbEB4\n+OGHCQ0NZfz48QwcOJDLLrsMs9ncoXKXAoFA0Nnx5hBab52JNi8ZWSwW5s+fz5NPPklVVRVRUVHI\n8gmtwCkQCAQnDW3JD5xSOYSDBw+SmpoKQH5+fsB7TS2w22J/LRAIBKcSitT6ktApFRDGjRvH5s2b\nARg5ciSSJB1S7rKt9tfHm7v+kUX63f9h/J038uO4y0gONjLpvmFMXJJDTd5Opjx0F1GfvcAnm4vp\nEWpi4PRxfFIs8dN3+dirions1o+EXqdx9ZmpJNrzOfjFegq2l1PocAMQa1JIiQ8hJjMZc88MjF0z\nqA2OJr/Ext7yevaXWrFWO3DU1uCqr8HtqA/IH4A3h6AYTX5RmmIyowSZ/YK0RlM7OUCUZjYqhJiU\nwEppsoxBkdtUKU1uIkiTpbaJ0vw1XY+xKK2t+YNjMdOedN9UnjlnIn979goGzPyAPpdcSa81T/Gf\nCjvzXp2M/cq/8f6NrxISk8zf/nQGtsVP8uXPpUQYZQaO7Yky7Fr++2U++dv3U1uwG0lWCEvqQbee\n0fyhezQxjlLsW7+jfOtBistsVLk1v6mdWZGJMirEBRmwJFmwJEVgSY1DiUlCCo9FC4miXvVgdevU\nu7yitBqnSo3NTY3d7Rel+Sql+aqm6brHnz9oTZQGCFHa8USSWs8RdOCLnJuby8yZM6muriYyMpL5\n8+fTpUuXgD66rvOPf/yDb775BlmWufnmm4+Jk8QRA4IvGAD88ssvR30wgUAgOFWQJWitGnxHftjM\nnj2byZMnM3bsWFauXMmsWbNYtGhRQJ+VK1eSn5/Pp59+SmVlJRMmTGDIkCEkJye3/4BNaHMS4NFH\nH21x+2OPPXZUAxAIBILOiNTG1h4qKyvZuXMnY8aMAWDs2LHs2LGDqqqqgH6rV6/m6quvBrxlN0eM\nGMGaNWuO4my8tDkg/O9//2tx+8qVK496EAKBQNDZOOQW78M0gKKiIg4ePBjQamtrD9lnUVERCQkJ\njUu2skx8fDzFxcUB/QoLCwNmA0lJSRQVFR31ObV6l5GvVJumaYeUbcvPzycyMvKoByEQCASdjbYs\nB/n6XHPNNRQUFAS8N3XqVKZNm3YcRtZxWg0IK1asAMDtdvufgzfpGBsby7x5847f6NrBG12vxF3/\nKm9fAHfckMeT/55EzRUP8sVVj5Ny9mieHt2LL/tdR5lT4/JRPZDG/ZV/vryJ0h3rMQRbSOuXyTln\nJHFht0hcn7xF3pd72FHrxK55iDDKdA81EpcZS2z/Hhi690ONSqXEprKvysaekjrqKu3YrU7cthpU\nRz2629VipTTZL0bzOp0aTMaGhLLsf5QVb0I5xKRgNil+cZrZpDSplIY/oex9bJsozT+eJgnlprR1\nytj079p6J8WJEqT5WOD5kJeAFedNo2zR43y/4J+8knIX41LDKct6gNve3Ezl/hzOnXQtl8fV89nT\nH1PoUBmbFEb6dX/imzIPazfmUZ2/E3d9DSExyST27MLo/klkxprx5HxO6Y+/UL6rkgK7So3b++9t\nVuQGUZqCJS6EsCQLIYkxmOLiMcQmoodEoZosWG0qdU6NGqdKld2bTK62ualzuLE7VG8y2aWhurzC\nNE1VAxxO9eaOp81EaUdKKIvk8dEjeTzgOXIWwfeNX7JkCZoW+JmHh4cf0j8pKYmSkhI8Hg+SJKHr\nOqWlpSQmJgb08/nJ9evXD/DOLJraCnWUVgPCm2++CcCzzz7LjBkzjvqAAoFAcEqgq96gcCQafgQl\nJSW1aZfR0dFkZGSQnZ1NVlYW2dnZZGZmEhUVFdBv1KhRLFu2jJEjR1JVVcXnn3/O4sWLO3QaTTni\nD8Kmt5hOnz4dXddbbAKBQPC7w6O3rbWTOXPmsHjxYkaNGsVbb73F3LlzAZgyZQrbt28HYPz48aSm\npnLxxRczceJE7rzzTr9m7Gg44gzhzDPP5KeffgIgMzPzkOUF37TmZNAhCAQCwQnFQ6tLRh1xM0pP\nT2fZsmWHbF+4cKH/uSzLzJkzp937bo0jBoSPPvrI//zzzz8/5gc/ljz/1Ov831O3s3zwJYxOtLBn\n1H3c9q9vcNXXMOfWc6mYfxercqsZEmOm39/u4NUtxezasB13fQ3xmUO4+LwujO6TQFTRZnav/oY9\neyopd2koEiQHG0noEkFc/24EnzYANTadGt3I/kore0qs5JfVe0VpNVW46mtQXYcxtWsiSpMNpkZR\nmknxG9oZjAoGg9wgRPPmDJoa23krpHlFaX5BmiL7xWhNRWk+IztfLqGpKK0pjdXUGsbbTlFac37r\nSmk+HvzL6zy79XWip77MqFuvp+quP1PocHPXu/O54N8b2bl2LTE9B/H8pIHk/3Man+XVkGY2csaN\n5+A4+3L+vXwrBdu93xHZYCKy2+kM6JvAH7pGE1a1j6ofN1KypZC8SjvlLg3N46uSJhNhVIgLMWJJ\nthCaFI0lJQ4lLgUi4tFDorC6NKwunUq7G6tLpc6lUml1UWN3YXWouJ0abqfaWCmtwdTOo2ktitKA\nNpvatYTIKXSADs4ATmaOGBCarns1T1g4HA5kWcZkMh2fkQkEAsFJjHQKBoQ26xDmzZvHzz//DMCX\nX37JOeecw9lnn83atWuP2+AEAoHg5KUt+YPOFTDaHBCys7Pp1asXAC+88AJPPfUUL730Es8+++xx\nG5xAIBCctHg8bUgqt5ZjOLloV8U0s9lMVVUV+fn5XHLJJQCHiC0EAoHgd4GmQmu5l1M1IHTr1o2V\nKw1MPAUAACAASURBVFeSl5fHkCFDAK/vRnBw8HEbXHtI7DeEMev/xexyG8//soTeT35B/sbVDJzw\nJ66NLOLfz36FWZG48LbzOZA+nH8/+zWV+3MIiUmm51k9mTQohd4WD5Wv/4/cLw6wr96NS/cQF6TQ\nw2IicUAC0WdkIKX1oVw1UmZzs7uinp1FtdRV2qmvteG0VqI66tGcdn+STpIVb1LZaEIxBXsTySYz\nBrMFY5ApUJRmkglq4mrqczk1N2wzyt5Esr9amuR7xO+A6heo0ehg2jyh3NzltCltSSgfT1HasdSu\nXT4wkeEfSxhDw3lvuIkZt27llqv7sNR4Fj9/NB9ddXHtnRPJzPucN1/7HrumM+HcFJJvupPXt5fy\n46aD1BXuwxBsISQ2mdRe8Yzum0DPMA+uL9dRvPEXSnZVUOhQsWvepQGLQSYuyEC0SSYs2UJ4ajhh\nXeIxxidjiEtBNUfhkIOodajUON3UNFRJq7S6qG4Qpjns7oaEsuZvmqoGJJS1hqppTUVpTRGitONP\nm3IIrRVdPsloc0CYPXs2jz/+OAaDgccffxyAb775xh8cBAKB4HeFb8noiH06VxGxNgeE/v37s3Tp\n0oBtWVlZZGVlHfNBCQQCwUlPG6wrTtklI4DvvvuOFStWUFpaSnx8PFlZWQwePPh4jU0gEAhOYtpy\n2+kpepfR8uXLmTFjBnFxcYwcOZL4+HjuvffeFhV1vwUfzxrGk/e8z19v/v/27jw8qvJ64Ph31sxk\nIXtCEiBIwh5QUChWLQKKokgq+oMoVKso3QS1WgpV1AJFgQqKokVFS1uLUItCsAjIYhQXRAVkE5El\nkAwkZJnJMvvc3x+TGbIxM8FAFs/neeYhc+fOnXsdzMv7nnvOGcjUb6LI/3Qd0V16s/I3Q/j8/j+w\n22zj5gEd6fjgk/x5w7cc2/EJACn9BjNpaAa91KXw+Tsc+d/XfHO6ilKHG6NGRUaEnqT+iSRf3ouw\nPoOxxXThuNnGwTNV7CuwcLqoCkupFbu5GGeVGae1svGktJrCdlq9saa4XVhNQpq6ThzhbFE7bZ2E\nNKNO448d6NTeZDRvh7R6sQOVCo3qbFE7X6e0QCuZtZPSIHBcIFj8oDWtmKb8bwOf/GM5qxZN4p0h\nv6B3VBiZb6zmT4s+QKPT033YLfz1pu588Yfn+KLMxpA4IwMevIWDhkyWfXCY4oM78bgcRHfqQcee\nWYy6Io2fdo5G8/3nFG/fyaldRRytclJak5Rm1KiI1WnoaNAQF2+kQ6coojonE56Wgja5C56IeDwR\n8VTY3VQ63JypdlJm9T5KKh2Yqx1U21w47DWxA4cHj8uD2+VNSvM93LUT00IoaldbqNtEcCrFE9Kj\nLQl5hvDaa6/xxhtv0KtXL/+2UaNGMXXqVH+jBiGE+NEIKajcTgeE8vJyMjIy6mzr1q0bZrO52U9K\nCCFaPY8bPK7A+1zkku8/VMhLRgMGDODpp5/GarUCUF1dzfz58xkwYMAFOzkhhGitftRLRrNmzeKR\nRx7hiiuuIDo6GrPZzIABA3j22WdDen95eTnTpk3jxIkT6PV60tPT+fOf/0xsbCy7du3iySefxG63\nk5aWxoIFC4iLizvvixJCiAvO4/E+AlE174Bgs9mYMWMG+/btQ6vVMm3aNK699toG+23evJklS5bg\ndDoBGDt2LPfcc0/Q4wcdEKxWKy+//DKHDh1i0KBBLFiwwH+XUf0uPoGoVCruv/9+Bg0aBMD8+fN5\n9tlnmTNnDtOmTWPevHkMGDCAl19+mb/+9a/+XIdQfTbi51waHQZz/s7y22fR5crR/PaOSwn/x0z+\n+9EJBsYY+MnTv+bf+Sq2rt9NdUkhCT0GMfJnl3BLjzic7z1P4Udf8+1OEyesTjQq6GzUkd6lAylX\nXEJE/4G4kntyssLJ/qJKjpVU812hNynNWl6KvaIUp63qnFVOfUlpGr3xbJVTnQZdmBZdmLZOQFmv\n1TSSlFa3yqmvS1qoVU59iWH1k9JU50g4g+arctoSSWkAP/3lC/z0rru55G+/57/FVTz7/kxufPlz\nTu/N46q7f8msMX0pmf8Q731RSEeDlqt+MQDVLVNZ+O4Bjn65F2vZKcLjU0nr050eGXHc0juZxKp8\nKj7ZQsHnRzhuquC03YXDo6BRQXRNQDkmJZLwhHCiL0kiqksy2o5dUMWl4IqIx+LwYHF4/AHlM5V2\nKm0uSqvslFc7sVu9CWn+wLLTm4zmcToarXLaWOJZ/ecSNL6ALvJtpcuWLSMyMpKNGzdy/PhxJkyY\nwKZNmzAajXX2S0xMZOnSpSQmJlJZWcnYsWPp378/l19+ecDjB10ymjVrFlu3bqVbt25s3LiRV199\nlf79+zdpMACIjo72DwYAl112GYWFhXzzzTeEhYX5l55ycnJYv359k44thBAX3QVqkBPI+vXrycnJ\nASA9PZ2srCzy8vIa7Ne/f38SExMBiIyMpFu3bhQWFgY9ftAB4aOPPmLZsmVMmzaNV199la1btzb1\nGhpQFIUVK1YwfPjwBr1Afa3iLBbLD/4cIYS4UFoihlBYWEhqaqr/eUpKCiaTKeB7vv/+e/bs2cOQ\nIUOCHj/oklF1dTVJSUn+D6+srAx60GBmzZpFREQEEydOZOPGjQ1eV84xDbNYLA0GCo1GE3K/UiGE\nAG9T+saa3jfW+P6cQrnLyKNp0ueNHTu2wS94X2fK7du3h35uNYqKivjd737Hk08+6Z8xBBJ0QHC7\n3Xz22Wf+X9Iul6vOc6BJ2crz5s0jPz+fpUuXAt5BpnbF1NLSUlQqVaNfzPLly3nxxRfrbEtLS2PL\nli3kmSp47Mu3yPjTejR6AyumD2Ng5W5efPJ/6NUqRj8yjKN9f85fn/2I4oOfEZHYmd5XZfGrK9OJ\n2LeRfau2YfryFHstdhwehY4GLT1jDKQOSiNh8KWouw3glNvAviILu06YOV5SRXlxFZXlVdgsxTir\nLQ2K2qm1erRhRn+XNJ0h0l/UThemRR+m8Re1Mxq0ZxPSNOo68QODVlOnqJ0vMU2jUtVsC17UDhrG\nD3x826B9FLXz0Rkj2XyDwtSsj5hy72Usj76Oz9+aT5crR7PinsuJ/fK/vL7oQ8xONxOvTSf9oT/y\nyq5TbNt2hLJje9EaIknufQXDr+jE1Rnx9Ix049y2iZN5eyjYd4YTVhdmp/dfgNE6DclhWhKjDcSk\nRxOeFElUlyT0aelok7vgikrEqjZgsbopqXZyptpBSbWDkkoHlXZXnaJ2tQvbuV0u3HZrnYS0+kXt\nfH/naiel+QSLJfxY4wsTJkxoUKn5gQceYMqUKaEfxOMOXu205vVQP2/16tUBD5eWlkZhYaF/JcVk\nMp3zX/4lJSXce++93H///f7q1MEEHRDi4+P505/+5H8eExNT57lKpQq5veaiRYvYv38/r7zyClqt\n96OzsrKw2+189dVXDBw4kLfeeotRo0Y1+v67776bW2+9tc42jUYT0mcLIYTPm2++2ei/2JtCUUK4\ny6hmyag5Pg/ghhtuYOXKlcyaNYtjx46xd+9eFi5c2GC/srIy7r33XiZOnMhtt90W8vGDDgjN1RHt\n8OHDvPLKK3Tt2pXx48cD0LlzZ1544QXmzZvHE088gcPhoFOnTixYsKDRYzR5SieEEI1olmVmjyeE\nGYKn+T4PmDRpEtOnT2fkyJFoNBpmz55NeHg4AIsXLyY5OZnx48fz6quvcvz4cVauXMlbb72FSqXi\nrrvuavAP6vqaVNzuh8jMzOTAgQONvjZgwAByc3Mv1qkIIcQP14Qlo+ZiNBp5/vnnG31t6tSp/p+n\nTZvGtGnTmnz8izYgXGgPzR7DiNXlmL7+gMeeeZTu7y9g07z1HKiw86vsHsQ+uogJf9vBkU8+QKM3\n0nXQEB4d2YOetiMc+/db7M87weFKB2anh2idmh6RetIGp5B6dX/0/a7GHJnGodNV7Cm0cKDATGW5\njYoyK7ayUzirLLhq1nl9fPEDjd6AWqtHZ4hEozeiMxjQh2nRhWn8cYSwMC1GvYZIgw6jTkOYVo2x\nJobQWFE7X2Oc2gXtGitqd66mOD7nKmrX2HJ+oPjBD80/uFB2vX4fC9MGcFNaB7RP/5PH7l1CeHwq\nSx68GtXS6Wxd+jG7zTZGp0Qx8InJfOhM5ZXcnZza+zEA8ZkDuezyVMZflkZGrB7V7nUUbv6Egh0m\nDlc6KLZ7A4qRWjUJeg2dw3XEdosh+pIEjEmxRHRNR5faFXdUMi5jHOZqF2VWF2eqHZypdlBssVNa\n5aDC5qSy2onD7sZhd+G0uXE53LgcTn/8wFOvoJ33eeCidhI/uLAUtxvcQf4bqtvWf+N2MyAIIcRF\npSghxBDacT8EIYQQXorLAbVWBRqlbls3vciAIIQQ58MTwgzBIzMEIYRo9xQlhKCyIjGEFvGP9Nv5\nZMEcrrnnHmbEfMtzv3+b76sc3NYrnn5LnmfKum/Z/f42XNZKugy5iV+P6cOIJDdFS1/l4Or97Dbb\nMDs9/i5pGZcl02loFuFXXIsjpQ/fF1n5ssDMV8fLKDlVgbXCQXXJaWzm4gZd0tRaPepaBe38XdKM\nkd5idgZNncCyUa8hqiagHF6TjOZLSgvTqjFovH/6uqTpNGcT0tQ1yWm+YLJGfbaoXf2Asv/8VHUD\nyrUT1YIlpF0IF+rwuwZeDcD1ezbSe8ZGqs8U8ticBxj6/X9ZOmcjhyrtDIwxMPSZ28jvM4Y///NL\njn6+HWeVmbhul5J5eTd+OSSdflEONPm7MG3cRH7eUb4trua03eXvkpag19ClJqAclxlPdEYaxuRE\ndGkZKHGdcEclUWZzY7a7OV3loKjKG1AuqrBTUmn3dkmzuXBYnTjrF7Wr1SWtsYQ0X0DZRwLKF1EL\n3GV0obWbAUEIIS4qWTISQggBNbOsYLedygxBCCF+BGTJqPV6YcEb9Bl1OxtuT2JF/8kU2pwMSwxn\n2FtPs+iwjndWbKTy9DE6XjqMO8f05u7+SVj//Rf2/WsHnxVXYXZ60Kiga7ieXj3i6Dy0F9FXDsXV\nZQCHy+zsLDSz82gppgIL5jPVOCpKsZadwlFlwe04W9ROrdXXiRv4E9Iiogkz6mpiB9qapDQtkQYt\nUQatvzmOsVYMwaBR12mKE+YvcKc+m4xW0xRHUxMHqN8Up35CWmPxA59QlvKbOyntQoYnPsg3M2/3\n6wxeuIuTX7zPz6fcz/SUQt6+9gUOVNjpGq5j9CPDcNw6jT+s3MO+bZ9RXVJIVEoGl1yexf1DuzEs\nvQOej/6Ned9ejm8+yMGjZk5YnVjdCnq1ilidN36Q0LkDcd1jienRmYhu3dDEp0ByV9wdOlLuhHKb\nG1OFnaIqO6ctNn/8oKLKgcvhwW511mmK43ZY6xS1qxs/cDeIH/ga5kDw+IDED5qJy4XicgbcReUK\nUg21lWk3A4IQQlxUcpeREEIIoEnF7doKGRCEEOI8eG/9DXaXUdsaEIK20BRCCNEIX1A52KMZ2Ww2\nHn74YUaOHMlNN93Etm3bAu7vcDi46aabuP3220M6fruZIXTMuoqNfxzChj4/47NSK78e34ce9+fw\npqs3i1/bTOmR3ST0GMSt2Zfx8FVd8Kx9jq9f3swnR8ootruJ1qlJ0GvpnxHDJdf3JmHYMJQeP+Vo\nlYrPT5TxyeEzHMs3U3a6kqrifFzWShzVdQPK/i5peiNaQwT6iGh0EdGotXp/MFlv1BJm0KEL0xBe\nE1CONOiIqgkwRxq0aNSqmmQ0b+VTg7ZWQppaXefP2gFlX7XTOj/TMKDsU79Lmk+oXdJac0AZYM57\njzFig4pv1q3iqrt/yZvXRfD+lfextbia7PRoet7al9hHF/HAuwf4bP3nWE4eIjw+lUsGXcEvR2Ry\nS8941DvXkL92A8X7T3NwfzHHqh1Uurw3IMTpNVwSoSOlUxQJPeOI651OZPdMdF17Q0Qsrg4pmN1a\nSq1uTJV2TlXaMZltmMw2iiw2zJUObFVO3G4PdqvrbLc0hx2P01GnS5qvwilA7S5pcO6AsgSPLzAl\nhCWjZu6pvGzZMiIjI9m4cSPHjx9nwoQJbNq0CaPR2Oj+ixYtYuDAgRw8eDCk48sMQQghzoPidof0\naE7r168nJycHgPT0dLKyssjLy2t03507d3L8+HGys7NDPn67mSEIIcRF5XJBkNtOaebbTgsLC0lN\nTfU/T0lJwWQyNdjParXy9NNP87e//Y2jR4+GfHwZEIQQ4jwoijv4slzNbacmk6nRnsr1WwKPHTu2\nwS94RVFQqVRs37495HObP38+EyZMIDExkSNHjoT8vnYzIGyYOYxt/a4k96SFX2X3IPON1aw5VMrM\nhZs5vTeP+MyBjL7tSp64LgP9hpf48rl1bN9/hkKbi2idmkujDXRMiSTjxj4k33A99L2Wo3Y9n5wo\n58NDxRw64i1qV1V8Erv5DC6HFZe1sk78QKM3ojNGojVE1PwZiT48Ao1Gjd6oI8yo9Re1iwzXNYgf\n+DukqVUN4gdhWnWD+EGjMYOaonaB4gf1t0PDonb1C9q1tfgBQPbhXnzyj2UMzvkFG3PS2Pyz/yP3\npIXRKVEM//s0GDSGh9Z9y//+u53SI7u98YOfXMWkUT0Z3zeRsK9zOfH2uxxe/x2nym0cqumop1FB\nYpiWruE60lKjSOyTQHxWVzr06o6+ay888el4jNFYMFBqc3ljBxV2TBYbpnJv/KDUYsdW7cRudeJx\nK3XiB2671ZuY5itu53T4O6QB/liCr0taU+IHEldoPk25y2jChAkUFBTUeemBBx5gypQpdbatXr06\n4OHS0tIoLCwkNjYW8A40Q4YMabDfl19+SV5eHkuWLMFut2M2m8nOzmbNmjUBj99uBgQhhLiYFLeC\n4g4yILi9xe3efPPNRmcITXXDDTewcuVKZs2axbFjx9i7dy8LFy5ssN/atWv9P+/YsYP58+fz9ttv\nBz2+DAhCCHEeFI8n+IBQM0NISUlpls+cNGkS06dPZ+TIkWg0GmbPnk14eDgAixcvJjk5mfHjx5/3\n8WVAEEKI89ASiWlGo5Hnn3++0demTp3a6PbBgweHNDsAGRCEEOL8uIPPEFTBZhCtTLsZED4b8XPW\nHi7lN//Xm4y/r2bUy5/z/VdHOb03j4Qegxg77ipmjczE8P6L7Pzru2zbU+QPKA+MMdDr2nTieqSS\nPOoGyBrOEZs3oLz1YBGHjpRxptBCxenj2M1ncFSZa4J8jQeUfQlp+vAIwgw61Fo1hgidPyEtMlxH\nTLiuQUKaUafxB49/SEBZo1b96APKAFtfXcZP77qbD8YlsWnIWNYcN3NLpw5c/9bj5CUNY0XuYd77\nT54/oJxx5dVMHt2bnKwkjF+tIX/lar5b9y27i6spc7obBJS7pEWR1C+RxEu7eQPKmf3xJHTFGdWR\nKqeH4moXhRV2Ciw2TpZbMZXbMJmtlFrsWCsd3gqnVhdutyekgLIvQQ0koNwauJ0uPI7At50qTql2\nKoQQ7V4oS0ZBl5RaGRkQhBDiPIQSVJYBQQghfgRCue1UcUtP5RaRZ6rg4d8Oxjj7DUb89WP2vPcu\nboeVjpcO4547BjHjZ+nY//0XPlm4gQ+/L6XY7iZOr2FQrIGeI7vR9ZZr0HXthaf3UA5VwMfHS9l2\nsIjvj5ZRYqqg8vTROvEDn9od0hqLHxgivDGEMIM3MS2qkYQ0X/wgvCaGoFapMGjrdkirHz/wdUqr\n3SGtfkG7szGBhh3SfPEG/3X8gPhBa4sd+Nz64K/59+Bq3h5wG1uLqxnXL4mhb83nP87u/PmlTyk+\nvB/LyUNEpWTQ4+ohTBnVk7G94uGjf3NkVS6H//c9X5VZOW134VZAr1aRoNeQGaknNT2a5H6JJPTP\nIKpnD/SZ/XHHdsYWkUhxlROrU+GkxUZBhY2TZVZvUbtyK+UVdmxVTmzVDpw2Nw67C4/bg9NW7e+Q\n5nJY/R3S6scPgnVIayxOILGDC0OWjIQQQgDeX/aeYHcZyYAghBDtn8fhxuMIfBeRytG2ZmcyIAgh\nxHlQlBCWjJq5H8KFJgOCEEKcByWExLSgpS1amXYzIDw0ewz7r7qPu5/YxNGP1xKVkkFi9yxm3DWQ\nOzu7KZr/EF/+bTt5Z6qpdHlINWgZnBpF99E9SRs1HM3Akdgjk9lbbGXbkRLyvi2mML+c0sISqksK\nsJvP4LRWNhpQ1hki0Boj0YfXBJSNRsKM3oBymFGHWqMizKgjOlxHlEFHZJivU5r3YdBqvEFkfyD5\nbIc0Q62kNG/C2dmAsqYmSqtReQO79QPKTUlGA9p0d7TGvBGVx8LBr1Boc/Kr7B5kLV3KM3udLP3n\nFk5/k4ficROfOZC+Qy9j+g09+Vm8E9fa5/h2xVYOfXKSvRY7xXY3GhUYNSpSDToyI3UkZcaR1C+Z\nhP6ZhPfsg65rH1yxnajUdqCk0puMVuFwU2CxcaK02h9Qrqpy1ASUnThqqps6bTYUj7tuQpo/Gc3d\nIKDsa7giAeWWJwOCEEIIwNunwBNkyUityG2nQgjR7skMQQghBEBIPZObu6fyhXbRBoR58+axceNG\nCgoKWLduHZmZmQAcO3aM6dOnU15eTkxMDPPnz6dLly5NPv4/0m9n1kN/p/zYXlIGXMfkCVdwc68k\nss7s4ODDz/PRe9+z12JDo1KR1SGMS/skkDnmMhJH3oS71884YPZw7JiZbd+d4cvDZyg6acFy6pQ3\nflBRhtth9a/HqrV6VGoN2jBvMpo3Ec0bPwgzeuMGeqPWn4ym1ar9xeyijd7ENKNe440faNR1uqOF\nadRo1Cp/DKF2/EBTExuoX8zuXPGDxpLR4McRPwB4bPwSUg06ps8djfKrZ8hesZvP126lwvQ9USkZ\nxHTpxajruzPlmq50rzzEmdf/ybf/3cneAyV8X+Wg0uVBr1aRHKYlWqcmI9ZAYp8Eki7tQlzWJYRl\n9keV1h1nTCfK3FpKKl3+ZLQKu4uCUisny6opspxNRrNbXTisTlxOtz8ZTfG4vcloLgeK2+2PH3hc\nzgbJaBBa/EBiBxeex+XGE6R4ncfVvN+DzWZjxowZ7Nu3D61Wy7Rp07j22msb3ffAgQP85S9/oays\nDJVKxR//+EeuueaagMe/aAPC9ddfzy9/+UvuvPPOOtuffPJJJk6cyOjRo1m7di0zZ85k+fLlF+u0\nhBDivLTEktGyZcuIjIxk48aNHD9+nAkTJrBp0yaMRmOd/axWK1OmTGHhwoX0798fj8dDRUVF0OOr\ng+7RTAYOHEhycjJKrSBLaWkpBw4c4OabbwZg9OjR7N+/n7Kysot1WkIIcV48NZnKAR/NnKm8fv16\ncnJyAEhPTycrK4u8vLwG+61bt44rrriC/v37A6BWq4mOjg56/BaNIZhMJpKTk88uaajVJCUlcerU\nKX8T6dosFgsWi6XONo1G02zt6YQQPw4mk6nRHsdN6nPsUULomKY03+cBhYWFpKam+p+npKRgMpka\n7Hf48GE0Gg2TJ0+muLiYvn37Mm3atKCf16aCysuXL+fFF1+ssy0tLY0tW7a00BkJIdqiCRMmUFBQ\nUGfbAw88wJQpU0I+RlOWjEL9vLFjxzb4Ba8oCiqViu3bt4d8bm63m88++4xVq1YRHx/P3LlzeeaZ\nZ5g7d27A97XogJCSksLp06f9F+zxeCgqKqJjx46N7n/33Xdz66231tmm0WgAeGHBG9jKHQwaP5Hn\n7hzAwMrdVL7/D7YFqG6qv/JmzkR1ZWd+JXmHz3C8pCpgdVOVWoNaq0cbZvT+eY7qpnqjN5gcFqYl\nJlyHXqsOWN3Ul4zm65CmVqnOWd3UF1g+VzAZqLMNGgaT62wLEEyu/56z+7XuYLLPLX0Sz1Y3fWoz\nxz7ZAEDaoJu4/rpMru2e4K9ueqiR6qbROjWdjToyYg0YE8JJ7pdIfN9L6NCnl7+6qTUikeJqF0VV\ndvLNNkwVNgpKrVTYXeesbuq0O/yJaB6Xw5+Mdq7qppKM1vzefPPNRv/F3hSKRwla3lqpmSGE+nmr\nV68OeLy0tDQKCwv9Kygmk4khQ4Y02C81NZUhQ4YQHx8PeJfjH3vssYDHhhYaEHxxhLi4OHr16kVu\nbi5jxowhNzeXPn36NLpcBOc3xRJCiPqaY5nZFycItk9zfR7ADTfcwMqVK5k1axbHjh1j7969LFy4\nsMF+o0aNYvLkyVRVVREREcFHH31Er169gh7/og0Ic+bMYdOmTZSUlHDPPfcQGxtLbm4uTz31FNOn\nT+ell14iOjqaefPmXaxTEkKI8+Z2eXA7g2Qqu5o3qDxp0iSmT5/OyJEj0Wg0zJ49m/DwcAAWL15M\ncnIy48ePJyUlhfvuu4+cnBzUajWdOnVi9uzZQY9/0QaExx9/nMcff7zB9m7durFq1aqLdRpCCNE8\nQuiYRjN3TDMajTz//PONvjZ16tQ6z7Ozs8nOzm7S8dtUUDmQjllX8dDoXkzqaaB82VO8//xWDpvt\nHKiwo1erGBhjoN+AZDJvGUjc9aNxZF7F7mIbH+45xdYDRZw8Xo610oHlVCHVJQU4qiwNktFqF7JT\na/X+ZDRDhN4bMzDUJKQZdUQatMSE64kM06LXqv3F7Ix6b+zAm5DmjR34OqP5Ctn5Ygi+mIFGfbYz\nWv1CdnDuZLRQYwe1t/u01WS02rpu2cyIf37F7vWvUFV8gpiuWfT52eU8emMvRqZq0JgOcGr+bA69\nu4s935dzrNqB1a1g1HiT0TIj9SR3iyGpXzLhSbHE98tEn9kfUjJxxHSixA4l5Q7yzTZOVdg4Uasz\nWrXNha3aia3Km4zmtLtw2F3eAnb1Ctn54wU1iWjSGa1taEoMoa1oNwOCEEJcTB63gifIgBDs9dZG\nBgQhhDgPUtxOCCEEAB5FwRNkScgj5a+FEKL9U9whxBBkyahlbJg5DPu/ZpM36WwimkYFvaPCGNA7\nnsxbLiXxhlF4eg9lfwV8vOsU2w4W+RPRqorzcVaZG61sqtbq0Rkj6ySiafTGRiubRoXragLItoym\nxwAAEmxJREFUZxPRvIlpjVc2rZ2A5vuzdgJa7cqmmiCJaFA3Ga2xYHLtfepvh/YRTPYZfM8SLCcP\nEZnclYG33cnvburF//X2JqIdeTaXkoMl7NtXzKHKs5VNUw3eYHJqenSdRDR1bBKaTj1wxXbxVjY1\nu/2VTU/WCib7EtFcTrc/mOztjOatbOpxObyVTWslogGNVjZtLJgcynNxcXicHtyOwP/tPUFuS21t\n2s2AIIQQF5PMEIQQQgBNy1RuK2RAEEKI86AoStA8A0WCyi3jk2uz2fDlIcxODx0NWm7p1IG47rF1\nEtH2FNv48Osz/kS0UlMZVcX52M1ncFor/UlB0DARzZeEpjd6YwdavfqciWi1k9Ai9Fpv0lmARDSd\nxlvQrnYRu3MlotVOQoPAiWjnihvUf632e+oLJXbQmuIGtam1en56193+RDT31n/y3YJNfLvlOHst\ndsxOtz8RrWu4rk4iWkL/DCJ69EKf0Q9XbCc8hmiK7VBicZFvrmyQiGapiR3YbU7sVhcelydgIlpj\nRexAEtHaEiWEPARZMhJCiB8BJYTSFTIgCCHEj4CUrhBCCAGA2+nG7Qy8bBfs9dZGBgQhhDgPHk8I\ntYyaeYZgs9mYMWMG+/btQ6vVMm3aNK699toG+ymKwty5c/n0009Rq9UkJyczd+5cEhMTAx6/3QwI\nHxdVkhGh59IrUsi4ZSCx192CJz6dQm0iWwssbF33LXsOl3CmwEJFkanRiqYqtQaN3lgnCU1riCQs\nKuZsF7SaQLJGqybcqCMmXEekQdcgmFy7oqlaVRNA1jRMRmusommgjmjnSkKDhsHkH0NF00B2LptM\n2qmdmN56nI/f2c03x8x1KpoaNWp6R+lIT/R2Q0vI6kJcVnf03fpCSiaumE6Y7FBidWEu9yahnSi3\ncrLUyimzlTMW+zkrmnqcjrNJaHWCyY1XNIWGHdHOFTSWYHIrEUIeQnOXv162bBmRkZFs3LiR48eP\nM2HCBDZt2oTRaKyz3+bNm/nmm2/Izc1FpVLxzDPP8PLLL/PEE08EPL464KtCCCEa5Y0heAI/mnmG\nsH79enJycgBIT08nKyuLvLy8BvupVCocDgdWqxWPx0NVVdU5WxPX1m5mCEIIcTE1pfy1yWRqtKdy\nU1sCFxYWkpqa6n+ekpKCyWRqsN/w4cPZsWMHV111FeHh4XTr1o0nn3wy6PFlQBBCiPPQlNIVEyZM\noKCgoM5rDzzwAFOmTKmzbezYsQ1+wSuKgkqlYvv27SGf2759+zhy5Agff/wx4eHh/OUvf+Hpp59m\n5syZAd/XbgaEh2aPIXLwz6hIuZQ9p6vJO1rCkT3l7D9yhDMFFiqLCqguKcRRZcbtsPrfp9bq0Roj\n0RkivF3QIqLRhUejj4hqEDcIM+qIDtcRZdCh16qJMeow6jVEGrREhmm9Retq4gZhWg06tTd24OuA\n5osb+IrU+eIGGnXDmAH88E5o9WMG9d9TW3uJG9T27VVDWXmighNWJw6PN26QatDRM0pPQo84IpIj\nSOzflZjeGegz+6MkZ+CKSaOw2kWp1UX+sQoKLDYKyqxU2FycMlspKrdhq3bisHoT0OxWbyE7l7US\nj8vhT0BTPO5zxg3qxwxq/ywJaG2HR1GClrf2vf7mm282OkOob/Xq1QGPl5aWRmFhIbGxsYB35jFk\nyJAG+73zzjsMGTKEiIgIAMaMGcNjjz0W8NggMQQhhDgvTkXB4Qn8cNYMCCkpKXTq1KnOo6nLRQA3\n3HADK1euBODYsWPs3buXa665psF+nTp14tNPP8XlcgHw4Ycf0r1796DHbzczBCGEuJjcioI7yAwh\n2OtNNWnSJKZPn87IkSPRaDTMnj2b8PBwABYvXkxycjLjx49nwoQJfPfdd4wZMwatVktqaiqzZs0K\nenwZEIQQ4jx4lOB3lTZ3orLRaOT5559v9LWpU6f6f9br9cydO7fJx283A8I/0m8nd6OJU/lbKTtV\nQlVRPm6H1d/wxket1aOLiEZniKj5MxJdRDSGiHDUWvU5YwaRYVpiws/GDPQ1sYLGYgZ14gVq1dlC\ndQFiBs1RtC7UPANonzGD+v73XSmpBh3DEsNJ7hlPUr+OJPTPxNijL9r03niM0bhi0ij2xQxKbRQc\nO0VBmZWTZVaKLDYqKry5Bh6XJ2DMwJd34I0VeALGDKRoXfvQEjOEC63dDAhCCHExtcQM4UKTAUEI\nIc6DWwk+A2hjxU5lQBBCiPPhVMARpCGaUwYEIYRo/ySG0Iq9sOANjh4/mwmo0RtRqTXeAnXxqd6E\ns4ho9OERdYLGYUYthpqOZ+F6DZEGHVFh3iJ1kQYtRp2mQfDYm2yGP3DclISz2gXpzpVwBoEL1YWa\ncBZq4Nh7/JB3bTPmvvso+m598SR0xR6ZTInVzXdVTvLNVkyn7JirnZws+5Yii41Six1rpQO71YnD\n6i1S57Q7/N3OFI/bGzyuE0CW4PGPmTuEGIIsGQkhxI+AJ4QZQrBM5tZGBgQhhDgPMkMQQggByAyh\nVeuYdRW6TBdhRq2/KJ1GoyYqXEeUQUu0UU9UTfOaSIOWCL0Wg7YmLqDxNq/RqesWoasdG9Co8Tez\n8a3haxpJLAMaxAvgbNOb2s9pZJ/6fkhsoD3GBZrijlOXU/6dHVvVYWzV+3Ha3NhtTlwOJ257TeMa\nXxMbfyE6iQ2I0MgMQQghBAAu5WzxukD7tCUyIAghxHlwE8Jtp8iAIIQQ7V57XDJqNf0Qjh07Rk5O\nDjfeeCM5OTnk5+e39CkJIcQ5+YLKgR7NHVReu3YtY8aMoW/fvrz55psB9121ahUjR45k5MiRzJkz\nJ6Tjt5oZwpNPPsnEiRMZPXo0a9euZebMmSxfvjzk92+YOQylja3XiQvrf0teaelTEO1YS8wQ+vTp\nw6JFi3j11VcD7nfy5EmWLFnCmjVriImJYdKkSaxZs4bs7OyA72sVM4TS0lIOHDjAzTffDMDo0aPZ\nv38/ZWVlLXxmQgjRuGCzg1BKWzRVZmYmGRkZde5ObMyGDRu4/vrriYmJAWDcuHGsX78+6PFbxYBg\nMplITk4+W9ZBrSYpKYlTp0618JkJIUTjfOWvAz1aqvy1yWQiNTXV/zwlJQWTyRT0fa1mySgUFosF\ni8VSZ5tGoyElJcV7v/6P/cZ7UUd6SnxLn4JoZdKSzjanb6zpfVP6HEekdsQZpNppRGrHJn3e2LFj\nG/ziVhQFlUrFJ598EnRm8EO1igEhJSWF06dP+y/c4/FQVFREx44d6+y3fPlyXnzxxTrbBg4cyIoV\nKwgLC7uYpyzagO/+t6ClT0G0Ur///e/56quv6mx74IEHmDJlSsjH+MPHb4e0n81mIzs7G7PZHPTz\nVq9eHfLnB5KSkkJBwdlinyaTiZSUlOBvVFqJX/ziF8qaNWsURVGUd999V7nrrrsa7GM2m5UTJ07U\neXzxxRdKTk6OUlhYeLFP+YIpLCxUhg0b1m6uSa6ndWtv16Moga+poqKiwe+REydOKGaz+YKcS2O/\nt37o502fPl3517/+dc7X8/PzlaFDhyqlpaWK2+1W7r33XuWdd94JetxWMUMAeOqpp5g+fTovvfQS\n0dHRzJs3r8E+55rSffXVVw2mY22Z2+2moKCg3VyTXE/r1t6uBwJfU2RkJJGRkRftXJq6FBXIe++9\nx/z587FYLGzZsoVXX32VZcuWkZGRweLFi0lOTmb8+PF07tyZ3/72t4wbNw6VSsXVV18d9A4jaCVL\nRgDdunVj1apVLX0aQgjRat18883+uzHrmzp1ap3n48aNY9y4cU06fqu4y0gIIUTLkwFBCCEEAJqn\nnnrqqZY+iR8qLCyMn/zkJ+3qTqP2dk1yPa1be7seaJ/XdKGpFEXqPQghhJAlIyGEEDVkQBBCCAG0\n8QGhLZbMnjdvHiNGjKBXr14cPnzYvz3QtbTm6ywvL2fy5MmMGjWK7Oxspk6d6i9KuGvXLrKzs7nx\nxhuZNGkSpaWl/vcFeq2l/e53v+PnP/85t956KxMnTuTgwYNA2/2OanvxxRfr/N1rq9/R8OHDuemm\nm/zf0/bt24G2ez2txnmnyrUCd911l5Kbm6soiqKsWbOm0ezm1ubLL79UTp06pQwfPlz57rvv/NsD\nXUtrvs7y8nJlx44d/ufz5s1THnvsMUVRFOX6669XvvrqK0VRFOWll15SZsyY4d8v0GstraKiwv/z\nBx98oNx6662KorTd78hn3759yn333acMGzbM/3evrX5Hw4cPVw4fPtxge1u9ntaizQ4IJSUlyqBB\ngxSPx6MoiqK43W7liiuuUEpLS1v4zEJT+3/KQNfS1q5zw4YNyj333KPs2bNHGT16tH97aWmpctll\nlymKogR8rbV55513lNtuu63Nf0d2u10ZP368cvLkSf/fvbb8HdX+/8enLV9Pa9FqMpWbKlDJ7NjY\n2BY+u6YJdC0ej6fNXKeiKKxYsYIRI0ZgMplIS0vzv+Y7V4vFEvC15krx/6Eef/xx/zLEa6+91ua/\no8WLF5OdnV3nv3tb/44effRRFEXh8ssv5+GHH27z19MatOkYgmhdZs2aRUREBBMnTmz0dSXAHc6B\nXmsJc+bMYevWrTz88MP+ulqt7RxDtWvXLr755hvuuOMO/7ZzXUtb+Y5WrFjBu+++y9tvv43H42HW\nrFmN7tdWrqe1aLMDQu2S2cA5S2a3BYGupa1c57x588jPz+e5554DGpbfLS0tRaVS0aFDh4CvtTZj\nxozh888/b9Pf0Y4dOzh69CgjRoxg+PDhnD59mvvuu4/8/Pw2+x0lJycDoNPpuPPOO/n6669JTU1t\ns9fTWrTZASEuLo5evXqRm5sLQG5uLn369Gk1U/RQ+H6BBLqWtnCdixYtYv/+/bz00ktotd5VyKys\nLOx2u7/m/FtvvcWoUaOCvtbSqqur63Tq27JlCzExMcTFxdG7d+82+R1NnjyZvLw8Nm/ezJYtW0hO\nTub1119n0qRJbfI7slqtVFZW+p+/99579OnTh759+7bJ62lN2nSm8pEjR5g+fToWi8VfMrtr164t\nfVoBzZkzh02bNlFSUkJMTAyxsbHk5uYGvJbWfJ2HDx/mlltuoWvXrv4SAZ07d+aFF17g66+/5okn\nnsDhcNCpUycWLFhAXFwc4F3GmDlzZqOvtaSSkhJ++9vfYrVaUavVxMTE8Mc//pHevXu32e+ovhEj\nRrB06VIyMzMDfg+t9Ts6ceIEU6dOxePx4PF4yMjI4PHHHychIaFNXk9r0qYHBCGEEM2nzS4ZCSGE\naF4yIAghhABkQBBCCFFDBgQhhBCADAhCCCFqyIAghBACkAFBCMBbTvnTTz9t6dMQokW12eJ2on0b\nPnw4JSUlaLVaNBoNGRkZZGdnM378eH8RufM1Y8YMOnbsyIMPPthMZytE+yADgmi1li5dypAhQ6is\nrOSLL75gzpw57N69m6effrqlT02IdkmWjESr5Uuij4yMZNiwYSxatIh3332Xw4cP43A4mDdvHsOG\nDePqq6/mqaeewuFwAN5ibkOHDvUPKCNGjPDXGVq1ahW5ubm89tprDBw4kN/85jf+zztw4ABjxoxh\n0KBB/P73v/cfT4gfC5khiDajf//+dOzYkZ07d7Jq1SpOnjzJ2rVr0Wg0PPLIIyxZsoSHH34YgDNn\nzlBeXs5HH33Erl27mDx5Mv369WPcuHF8/fXXjS4Zvf/++7z++uvo9XpycnJ45513GD9+fEtcqhAt\nQmYIok1JSkqivLyc//znP8yYMYOoqCjCw8OZPHky69at8++nUql46KGH0Ol0DBo0iKFDh7J+/fqA\nx77rrrtISEigQ4cODBs2jAMHDlzoyxGiVZEZgmhTTp8+jdvtxmq1ctttt/m3ezyeOg1POnTo4K++\nCpCamkpRUVHAY8fHx/t/NhqNFBcXN+OZC9H6yYAg2ow9e/ZQVFTEddddx2uvvca6detISkpqdF+L\nxYLNZsNgMADedpE9evS4mKcrRJsjS0ai1ausrGTr1q088sgjZGdn07NnT26//Xbmzp1LaWkp4J05\nfPzxx/73KIrC4sWLcTqd7Ny5k23btvkboiQkJHDixIkWuRYhWjOZIYhW6ze/+Q0ajQa1Wk1GRgb3\n3nsvOTk5APzhD39gyZIljBs3jvLycpKTk7njjju4+uqrAUhMTCQ6OpprrrmG8PBwZs2a5W9Yc/vt\nt/Pggw8yePBgBg8ezIsvvviDcxuEaA+kQY5od3bs2MG0adPYtm1bS5+KEG2KLBkJIYQAZEAQQghR\nQ5aMhBBCADJDEEIIUUMGBCGEEIAMCEIIIWrIgCCEEAKQAUEIIUQNGRCEEEIA8P9U11JhKH720wAA\nAABJRU5ErkJggg==\n",
            "text/plain": [
              "\u003cmatplotlib.figure.Figure at 0x7fa7c533cd90\u003e"
            ]
          },
          "metadata": {
            "tags": []
          },
          "output_type": "display_data"
        }
      ],
      "source": [
        "pos_encoding = positional_encoding(50, 512)\n",
        "print (pos_encoding.shape)\n",
        "\n",
        "plt.pcolormesh(pos_encoding[0], cmap='RdBu')\n",
        "plt.xlabel('Depth')\n",
        "plt.xlim((0, 512))\n",
        "plt.ylabel('Position')\n",
        "plt.colorbar()\n",
        "plt.show()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "a_b4ou4TYqUN"
      },
      "source": [
        "## Masking"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "s42Uydjkv0hF"
      },
      "source": [
        "Mask all the pad tokens in the batch of sequence. It ensures that the model does not treat padding as the input. The mask indicates where pad value `0` is present: it outputs a `1` at those locations, and a `0` otherwise."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 0,
      "metadata": {
        "colab": {},
        "colab_type": "code",
        "id": "U2i8-e1s8ti9"
      },
      "outputs": [],
      "source": [
        "def create_padding_mask(seq):\n",
        "  seq = tf.cast(tf.math.equal(seq, 0), tf.float32)\n",
        "  \n",
        "  # add extra dimensions so that we can add the padding\n",
        "  # to the attention logits.\n",
        "  return seq[:, tf.newaxis, tf.newaxis, :]  # (batch_size, 1, 1, seq_len)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 0,
      "metadata": {
        "colab": {
          "height": 161
        },
        "colab_type": "code",
        "id": "A7BYeBCNvi7n",
        "outputId": "74c22d7e-6d7e-49ab-9dc1-ef67d3155bd7"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "\u003ctf.Tensor: id=311505, shape=(3, 1, 1, 5), dtype=float32, numpy=\n",
              "array([[[[ 0.,  0.,  1.,  1.,  0.]]],\n",
              "\n",
              "\n",
              "       [[[ 0.,  0.,  0.,  1.,  1.]]],\n",
              "\n",
              "\n",
              "       [[[ 1.,  1.,  1.,  0.,  0.]]]], dtype=float32)\u003e"
            ]
          },
          "execution_count": 0,
          "metadata": {
            "tags": []
          },
          "output_type": "execute_result"
        }
      ],
      "source": [
        "x = tf.constant([[7, 6, 0, 0, 1], [1, 2, 3, 0, 0], [0, 0, 0, 4, 5]])\n",
        "create_padding_mask(x)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "Z0hzukDBgVom"
      },
      "source": [
        "The look-ahead mask is used to mask the future tokens in a sequence. In other words, the mask indicates which entries should not be used.\n",
        "\n",
        "This means that to predict the third word, only the first and second word will be used. Similarly to predict the fourth word, only the first, second and the third word will be used and so on."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 0,
      "metadata": {
        "colab": {},
        "colab_type": "code",
        "id": "dVxS8OPI9uI0"
      },
      "outputs": [],
      "source": [
        "def create_look_ahead_mask(size):\n",
        "  mask = 1 - tf.linalg.band_part(tf.ones((size, size)), -1, 0)\n",
        "  return mask  # (seq_len, seq_len)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 0,
      "metadata": {
        "colab": {
          "height": 89
        },
        "colab_type": "code",
        "id": "yxKGuXxaBeeE",
        "outputId": "bbe2d22d-e2ef-458e-e198-f57317380c08"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "\u003ctf.Tensor: id=311521, shape=(3, 3), dtype=float32, numpy=\n",
              "array([[ 0.,  1.,  1.],\n",
              "       [ 0.,  0.,  1.],\n",
              "       [ 0.,  0.,  0.]], dtype=float32)\u003e"
            ]
          },
          "execution_count": 0,
          "metadata": {
            "tags": []
          },
          "output_type": "execute_result"
        }
      ],
      "source": [
        "x = tf.random.uniform((1, 3))\n",
        "temp = create_look_ahead_mask(x.shape[1])\n",
        "temp"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "xluDl5cXYy4y"
      },
      "source": [
        "## Scaled dot product attention"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "vsxEE_-Wa1gF"
      },
      "source": [
        "\u003cimg src=\"https://www.tensorflow.org/images/tutorials/transformer/scaled_attention.png\" width=\"500\" alt=\"scaled_dot_product_attention\"\u003e\n",
        "\n",
        "The attention function used by the transformer takes three inputs: Q (query), K (key), V (value). The equation used to calculate the attention weights is:\n",
        "\n",
        "$$\\Large{Attention(Q, K, V) = softmax_k(\\frac{QK^T}{\\sqrt{d_k}}) V} $$\n",
        "\n",
        "The dot-product attention is scaled by a factor of square root of the depth. This is done because for large values of depth, the dot product grows large in magnitude pushing the softmax function where it has small gradients resulting in a very hard softmax. \n",
        "\n",
        "For example, consider that `Q` and `K` have a mean of 0 and variance of 1. Their matrix multiplication will have a mean of 0 and variance of `dk`. Hence, *square root of `dk`* is used for scaling (and not any other number) because the matmul of `Q` and `K` should have a mean of 0 and variance of 1, so that we get a gentler softmax.\n",
        "\n",
        "The mask is multiplied with *-1e9 (close to negative infinity).* This is done because the mask is summed with the scaled matrix multiplication of Q and K and is applied immediately before a softmax. The goal is to zero out these cells, and large negative inputs to softmax are near zero in the output."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 0,
      "metadata": {
        "colab": {},
        "colab_type": "code",
        "id": "LazzUq3bJ5SH"
      },
      "outputs": [],
      "source": [
        "def scaled_dot_product_attention(q, k, v, mask):\n",
        "  \"\"\"Calculate the attention weights.\n",
        "  q, k, v must have matching leading dimensions.\n",
        "  k, v must have matching penultimate dimension, i.e.: seq_len_k = seq_len_v.\n",
        "  The mask has different shapes depending on its type(padding or look ahead) \n",
        "  but it must be broadcastable for addition.\n",
        "  \n",
        "  Args:\n",
        "    q: query shape == (..., seq_len_q, depth)\n",
        "    k: key shape == (..., seq_len_k, depth)\n",
        "    v: value shape == (..., seq_len_v, depth_v)\n",
        "    mask: Float tensor with shape broadcastable \n",
        "          to (..., seq_len_q, seq_len_k). Defaults to None.\n",
        "    \n",
        "  Returns:\n",
        "    output, attention_weights\n",
        "  \"\"\"\n",
        "\n",
        "  matmul_qk = tf.matmul(q, k, transpose_b=True)  # (..., seq_len_q, seq_len_k)\n",
        "  \n",
        "  # scale matmul_qk\n",
        "  dk = tf.cast(tf.shape(k)[-1], tf.float32)\n",
        "  scaled_attention_logits = matmul_qk / tf.math.sqrt(dk)\n",
        "\n",
        "  # add the mask to the scaled tensor.\n",
        "  if mask is not None:\n",
        "    scaled_attention_logits += (mask * -1e9)\n",
        "\n",
        "  # softmax is normalized on the last axis (seq_len_k) so that the scores\n",
        "  # add up to 1.\n",
        "  attention_weights = tf.nn.softmax(scaled_attention_logits, axis=-1)  # (..., seq_len_q, seq_len_k)\n",
        "\n",
        "  output = tf.matmul(attention_weights, v)  # (..., seq_len_v, depth_v)\n",
        "\n",
        "  return output, attention_weights"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "FiqETnhCkoXh"
      },
      "source": [
        "As the softmax normalization is done on K, its values decide the amount of importance given to Q.\n",
        "\n",
        "The output represents the multiplication of the attention weights and the V (value) vector. This ensures that the words we want to focus on are kept as is and the irrelevant words are flushed out."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 0,
      "metadata": {
        "colab": {},
        "colab_type": "code",
        "id": "n90YjClyInFy"
      },
      "outputs": [],
      "source": [
        "def print_out(q, k, v):\n",
        "  temp_out, temp_attn = scaled_dot_product_attention(\n",
        "      q, k, v, None)\n",
        "  print ('Attention weights are:')\n",
        "  print (temp_attn)\n",
        "  print ('Output is:')\n",
        "  print (temp_out)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 0,
      "metadata": {
        "colab": {
          "height": 89
        },
        "colab_type": "code",
        "id": "yAzUAf2DPlNt",
        "outputId": "24653f6f-b337-455c-85d9-3b87f7d3fe9c"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Attention weights are:\n",
            "tf.Tensor([[ 0.  1.  0.  0.]], shape=(1, 4), dtype=float32)\n",
            "Output is:\n",
            "tf.Tensor([[ 10.   0.]], shape=(1, 2), dtype=float32)\n"
          ]
        }
      ],
      "source": [
        "np.set_printoptions(suppress=True)\n",
        "\n",
        "temp_k = tf.constant([[10,0,0],\n",
        "                      [0,10,0],\n",
        "                      [0,0,10],\n",
        "                      [0,0,10]], dtype=tf.float32)  # (4, 3)\n",
        "\n",
        "temp_v = tf.constant([[   1,0],\n",
        "                      [  10,0],\n",
        "                      [ 100,5],\n",
        "                      [1000,6]], dtype=tf.float32)  # (4, 2)\n",
        "\n",
        "# This `query` aligns with the second `key`,\n",
        "# so the second `value` is returned.\n",
        "temp_q = tf.constant([[0, 10, 0]], dtype=tf.float32)  # (1, 3)\n",
        "print_out(temp_q, temp_k, temp_v)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 0,
      "metadata": {
        "colab": {
          "height": 89
        },
        "colab_type": "code",
        "id": "zg6k-fGhgXra",
        "outputId": "5e0fd609-5fcd-4f95-a237-56ed23fffd61"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Attention weights are:\n",
            "tf.Tensor([[ 0.   0.   0.5  0.5]], shape=(1, 4), dtype=float32)\n",
            "Output is:\n",
            "tf.Tensor([[ 550.     5.5]], shape=(1, 2), dtype=float32)\n"
          ]
        }
      ],
      "source": [
        "# This query aligns with a repeated key (third and fourth), \n",
        "# so all associated values get averaged.\n",
        "temp_q = tf.constant([[0, 0, 10]], dtype=tf.float32)  # (1, 3)\n",
        "print_out(temp_q, temp_k, temp_v)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 0,
      "metadata": {
        "colab": {
          "height": 89
        },
        "colab_type": "code",
        "id": "UAq3YOzUgXhb",
        "outputId": "a5f8f528-311b-4c4d-d44e-c6b2fd2ae6c6"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Attention weights are:\n",
            "tf.Tensor([[ 0.5  0.5  0.   0. ]], shape=(1, 4), dtype=float32)\n",
            "Output is:\n",
            "tf.Tensor([[ 5.5  0. ]], shape=(1, 2), dtype=float32)\n"
          ]
        }
      ],
      "source": [
        "# This query aligns equally with the first and second key, \n",
        "# so their values get averaged.\n",
        "temp_q = tf.constant([[10, 10, 0]], dtype=tf.float32)  # (1, 3)\n",
        "print_out(temp_q, temp_k, temp_v)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "aOz-4_XIhaTP"
      },
      "source": [
        "Pass all the queries together."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 0,
      "metadata": {
        "colab": {
          "height": 197
        },
        "colab_type": "code",
        "id": "6dlU8Tm-hYrF",
        "outputId": "47feec61-8321-41d3-e7dc-c715fc48c4c0"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Attention weights are:\n",
            "tf.Tensor(\n",
            "[[ 0.   0.   0.5  0.5]\n",
            " [ 0.   1.   0.   0. ]\n",
            " [ 0.5  0.5  0.   0. ]], shape=(3, 4), dtype=float32)\n",
            "Output is:\n",
            "tf.Tensor(\n",
            "[[ 550.     5.5]\n",
            " [  10.     0. ]\n",
            " [   5.5    0. ]], shape=(3, 2), dtype=float32)\n"
          ]
        }
      ],
      "source": [
        "temp_q = tf.constant([[0, 0, 10], [0, 10, 0], [10, 10, 0]], dtype=tf.float32)  # (3, 3)\n",
        "print_out(temp_q, temp_k, temp_v)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "kmzGPEy64qmA"
      },
      "source": [
        "## Multi-head attention"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "fz5BMC8Kaoqo"
      },
      "source": [
        "\u003cimg src=\"https://www.tensorflow.org/images/tutorials/transformer/multi_head_attention.png\" width=\"500\" alt=\"multi-head attention\"\u003e\n",
        "\n",
        "\n",
        "Multi-head attention consists of four parts:\n",
        "*    Linear layers and split into heads.\n",
        "*    Scaled dot-product attention.\n",
        "*    Concatenation of heads.\n",
        "*    Final linear layer."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "JPmbr6F1C-v_"
      },
      "source": [
        "Each multi-head attention block gets three inputs; Q (query), K (key), V (value). These are put through linear (Dense) layers and split up into multiple heads. \n",
        "\n",
        "The `scaled_dot_product_attention` defined above is applied to each head (broadcasted for efficiency). An appropriate mask must be used in the attention step.  The attention output for each head is then concatenated (using `tf.transpose`, and `tf.reshape`) and put through a final `Dense` layer.\n",
        "\n",
        "Instead of one single attention head, Q, K, and V are split into multiple heads because it allows the model to jointly attend to information at different positions from different representational spaces. After the split each head has a reduced dimensionality, so the total computation cost is the same as a single head attention with full dimensionality."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 0,
      "metadata": {
        "colab": {},
        "colab_type": "code",
        "id": "757ll3CdWLh8"
      },
      "outputs": [],
      "source": [
        "class MultiHeadAttention(tf.keras.layers.Layer):\n",
        "  def __init__(self, d_model, num_heads):\n",
        "    super(MultiHeadAttention, self).__init__()\n",
        "    self.num_heads = num_heads\n",
        "    self.d_model = d_model\n",
        "    \n",
        "    assert d_model % self.num_heads == 0\n",
        "    \n",
        "    self.depth = d_model // self.num_heads\n",
        "    \n",
        "    self.wq = tf.keras.layers.Dense(d_model)\n",
        "    self.wk = tf.keras.layers.Dense(d_model)\n",
        "    self.wv = tf.keras.layers.Dense(d_model)\n",
        "    \n",
        "    self.dense = tf.keras.layers.Dense(d_model)\n",
        "        \n",
        "  def split_heads(self, x, batch_size):\n",
        "    \"\"\"Split the last dimension into (num_heads, depth).\n",
        "    Transpose the result such that the shape is (batch_size, num_heads, seq_len, depth)\n",
        "    \"\"\"\n",
        "    x = tf.reshape(x, (batch_size, -1, self.num_heads, self.depth))\n",
        "    return tf.transpose(x, perm=[0, 2, 1, 3])\n",
        "    \n",
        "  def call(self, v, k, q, mask):\n",
        "    batch_size = tf.shape(q)[0]\n",
        "    \n",
        "    q = self.wq(q)  # (batch_size, seq_len, d_model)\n",
        "    k = self.wk(k)  # (batch_size, seq_len, d_model)\n",
        "    v = self.wv(v)  # (batch_size, seq_len, d_model)\n",
        "    \n",
        "    q = self.split_heads(q, batch_size)  # (batch_size, num_heads, seq_len_q, depth)\n",
        "    k = self.split_heads(k, batch_size)  # (batch_size, num_heads, seq_len_k, depth)\n",
        "    v = self.split_heads(v, batch_size)  # (batch_size, num_heads, seq_len_v, depth)\n",
        "    \n",
        "    # scaled_attention.shape == (batch_size, num_heads, seq_len_v, depth)\n",
        "    # attention_weights.shape == (batch_size, num_heads, seq_len_q, seq_len_k)\n",
        "    scaled_attention, attention_weights = scaled_dot_product_attention(\n",
        "        q, k, v, mask)\n",
        "    \n",
        "    scaled_attention = tf.transpose(scaled_attention, perm=[0, 2, 1, 3])  # (batch_size, seq_len_v, num_heads, depth)\n",
        "\n",
        "    concat_attention = tf.reshape(scaled_attention, \n",
        "                                  (batch_size, -1, self.d_model))  # (batch_size, seq_len_v, d_model)\n",
        "\n",
        "    output = self.dense(concat_attention)  # (batch_size, seq_len_v, d_model)\n",
        "        \n",
        "    return output, attention_weights"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "0D8FJue5lDyZ"
      },
      "source": [
        "Create a `MultiHeadAttention` layer to try out. At each location in the sequence, `y`, the `MultiHeadAttention` runs all 8 attention heads across all other locations in the sequence, returning a new vector of the same length at each location."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 0,
      "metadata": {
        "colab": {
          "height": 35
        },
        "colab_type": "code",
        "id": "Hu94p-_-2_BX",
        "outputId": "719aeb5f-721e-43ac-e8a8-e38ea65771b4"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "(TensorShape([1, 60, 512]), TensorShape([1, 8, 60, 60]))"
            ]
          },
          "execution_count": 0,
          "metadata": {
            "tags": []
          },
          "output_type": "execute_result"
        }
      ],
      "source": [
        "temp_mha = MultiHeadAttention(d_model=512, num_heads=8)\n",
        "y = tf.random.uniform((1, 60, 512))  # (batch_size, encoder_sequence, d_model)\n",
        "out, attn = temp_mha(y, k=y, q=y, mask=None)\n",
        "out.shape, attn.shape"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "RdDqGayx67vv"
      },
      "source": [
        "## Point wise feed forward network"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "gBqzJXGfHK3X"
      },
      "source": [
        "Point wise feed forward network consists of two fully-connected layers with a ReLU activation in between."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 0,
      "metadata": {
        "colab": {},
        "colab_type": "code",
        "id": "ET7xLt0yCT6Z"
      },
      "outputs": [],
      "source": [
        "def point_wise_feed_forward_network(d_model, dff):\n",
        "  return tf.keras.Sequential([\n",
        "      tf.keras.layers.Dense(dff, activation='relu'),  # (batch_size, seq_len, dff)\n",
        "      tf.keras.layers.Dense(d_model)  # (batch_size, seq_len, d_model)\n",
        "  ])"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 0,
      "metadata": {
        "colab": {
          "height": 35
        },
        "colab_type": "code",
        "id": "mytb1lPyOHLB",
        "outputId": "559df501-6254-4b2e-d057-a209658e6961"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "TensorShape([64, 50, 512])"
            ]
          },
          "execution_count": 0,
          "metadata": {
            "tags": []
          },
          "output_type": "execute_result"
        }
      ],
      "source": [
        "sample_ffn = point_wise_feed_forward_network(512, 2048)\n",
        "sample_ffn(tf.random.uniform((64, 50, 512))).shape"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "7e7hKcxn6-zd"
      },
      "source": [
        "## Encoder and decoder"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "yScbC0MUH8dS"
      },
      "source": [
        "\u003cimg src=\"https://www.tensorflow.org/images/tutorials/transformer/transformer.png\" width=\"600\" alt=\"transformer\"\u003e"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "MfYJG-Kvgwy2"
      },
      "source": [
        "The transformer model follows the same general pattern as a standard [sequence to sequence with attention model](nmt_with_attention.ipynb). \n",
        "\n",
        "* The input sentence is passed through `N` encoder layers that generates an output for each word/token in the sequence.\n",
        "* The decoder attends on the encoder's output and its own input (self-attention) to predict the next word. "
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "QFv-FNYUmvpn"
      },
      "source": [
        "### Encoder layer\n",
        "\n",
        "Each encoder layer consists of sublayers:\n",
        "\n",
        "1.   Multi-head attention (with padding mask) \n",
        "2.    Point wise feed forward networks. \n",
        "\n",
        "Each of these sublayers has a residual connection around it followed by a layer normalization. Residual connections help in avoiding the vanishing gradient problem in deep networks.\n",
        "\n",
        "The output of each sublayer is `LayerNorm(x + Sublayer(x))`. The normalization is done on the `d_model` (last) axis. There are N encoder layers in the transformer."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 0,
      "metadata": {
        "colab": {},
        "colab_type": "code",
        "id": "ncyS-Ms3i2x_"
      },
      "outputs": [],
      "source": [
        "class EncoderLayer(tf.keras.layers.Layer):\n",
        "  def __init__(self, d_model, num_heads, dff, rate=0.1):\n",
        "    super(EncoderLayer, self).__init__()\n",
        "\n",
        "    self.mha = MultiHeadAttention(d_model, num_heads)\n",
        "    self.ffn = point_wise_feed_forward_network(d_model, dff)\n",
        "\n",
        "    self.layernorm1 = tf.keras.layers.LayerNormalization(epsilon=1e-6)\n",
        "    self.layernorm2 = tf.keras.layers.LayerNormalization(epsilon=1e-6)\n",
        "    \n",
        "    self.dropout1 = tf.keras.layers.Dropout(rate)\n",
        "    self.dropout2 = tf.keras.layers.Dropout(rate)\n",
        "    \n",
        "  def call(self, x, training, mask):\n",
        "\n",
        "    attn_output, _ = self.mha(x, x, x, mask)  # (batch_size, input_seq_len, d_model)\n",
        "    attn_output = self.dropout1(attn_output, training=training)\n",
        "    out1 = self.layernorm1(x + attn_output)  # (batch_size, input_seq_len, d_model)\n",
        "    \n",
        "    ffn_output = self.ffn(out1)  # (batch_size, input_seq_len, d_model)\n",
        "    ffn_output = self.dropout2(ffn_output, training=training)\n",
        "    out2 = self.layernorm2(out1 + ffn_output)  # (batch_size, input_seq_len, d_model)\n",
        "    \n",
        "    return out2"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 0,
      "metadata": {
        "colab": {
          "height": 35
        },
        "colab_type": "code",
        "id": "AzZRXdO0mI48",
        "outputId": "e5992b5b-6f13-4868-ee5c-f055f0c61329"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "TensorShape([64, 43, 512])"
            ]
          },
          "execution_count": 0,
          "metadata": {
            "tags": []
          },
          "output_type": "execute_result"
        }
      ],
      "source": [
        "sample_encoder_layer = EncoderLayer(512, 8, 2048)\n",
        "\n",
        "sample_encoder_layer_output = sample_encoder_layer(\n",
        "    tf.random.uniform((64, 43, 512)), False, None)\n",
        "\n",
        "sample_encoder_layer_output.shape  # (batch_size, input_seq_len, d_model)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "6LO_48Owmx_o"
      },
      "source": [
        "### Decoder layer\n",
        "\n",
        "Each decoder layer consists of sublayers:\n",
        "\n",
        "1.   Masked multi-head attention (with look ahead mask and padding mask)\n",
        "2.   Multi-head attention (with padding mask). V (value) and K (key) receive the *encoder output* as inputs. Q (query) receives the *output from the masked multi-head attention sublayer.*\n",
        "3.   Point wise feed forward networks\n",
        "\n",
        "Each of these sublayers has a residual connection around it followed by a layer normalization. The output of each sublayer is `LayerNorm(x + Sublayer(x))`. The normalization is done on the `d_model` (last) axis.\n",
        "\n",
        "There are N decoder layers in the transformer.\n",
        "\n",
        "As Q receives the output from decoder's first attention block, and K receives the encoder output, the attention weights represent the importance given to the decoder's input based on the encoder's output. In other words, the decoder predicts the next word by looking at the encoder output and self-attending to its own output. See the demonstration above in the scaled dot product attention section."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 0,
      "metadata": {
        "colab": {},
        "colab_type": "code",
        "id": "9SoX0-vd1hue"
      },
      "outputs": [],
      "source": [
        "class DecoderLayer(tf.keras.layers.Layer):\n",
        "  def __init__(self, d_model, num_heads, dff, rate=0.1):\n",
        "    super(DecoderLayer, self).__init__()\n",
        "\n",
        "    self.mha1 = MultiHeadAttention(d_model, num_heads)\n",
        "    self.mha2 = MultiHeadAttention(d_model, num_heads)\n",
        "\n",
        "    self.ffn = point_wise_feed_forward_network(d_model, dff)\n",
        " \n",
        "    self.layernorm1 = tf.keras.layers.LayerNormalization(epsilon=1e-6)\n",
        "    self.layernorm2 = tf.keras.layers.LayerNormalization(epsilon=1e-6)\n",
        "    self.layernorm3 = tf.keras.layers.LayerNormalization(epsilon=1e-6)\n",
        "    \n",
        "    self.dropout1 = tf.keras.layers.Dropout(rate)\n",
        "    self.dropout2 = tf.keras.layers.Dropout(rate)\n",
        "    self.dropout3 = tf.keras.layers.Dropout(rate)\n",
        "    \n",
        "    \n",
        "  def call(self, x, enc_output, training, \n",
        "           look_ahead_mask, padding_mask):\n",
        "    # enc_output.shape == (batch_size, input_seq_len, d_model)\n",
        "\n",
        "    attn1, attn_weights_block1 = self.mha1(x, x, x, look_ahead_mask)  # (batch_size, target_seq_len, d_model)\n",
        "    attn1 = self.dropout1(attn1, training=training)\n",
        "    out1 = self.layernorm1(attn1 + x)\n",
        "    \n",
        "    attn2, attn_weights_block2 = self.mha2(\n",
        "        enc_output, enc_output, out1, padding_mask)  # (batch_size, target_seq_len, d_model)\n",
        "    attn2 = self.dropout2(attn2, training=training)\n",
        "    out2 = self.layernorm2(attn2 + out1)  # (batch_size, target_seq_len, d_model)\n",
        "    \n",
        "    ffn_output = self.ffn(out2)  # (batch_size, target_seq_len, d_model)\n",
        "    ffn_output = self.dropout3(ffn_output, training=training)\n",
        "    out3 = self.layernorm3(ffn_output + out2)  # (batch_size, target_seq_len, d_model)\n",
        "    \n",
        "    return out3, attn_weights_block1, attn_weights_block2"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 0,
      "metadata": {
        "colab": {
          "height": 35
        },
        "colab_type": "code",
        "id": "Ne2Bqx8k71l0",
        "outputId": "6cc2fe6b-8570-403d-eb27-c41d9018fa4e"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "TensorShape([64, 50, 512])"
            ]
          },
          "execution_count": 0,
          "metadata": {
            "tags": []
          },
          "output_type": "execute_result"
        }
      ],
      "source": [
        "sample_decoder_layer = DecoderLayer(512, 8, 2048)\n",
        "\n",
        "sample_decoder_layer_output, _, _ = sample_decoder_layer(\n",
        "    tf.random.uniform((64, 50, 512)), sample_encoder_layer_output, \n",
        "    False, None, None)\n",
        "\n",
        "sample_decoder_layer_output.shape  # (batch_size, target_seq_len, d_model)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "SE1H51Ajm0q1"
      },
      "source": [
        "### Encoder\n",
        "\n",
        "The `Encoder` consists of:\n",
        "1.   Input Embedding\n",
        "2.   Positional Encoding\n",
        "3.   N encoder layers\n",
        "\n",
        "The input is put through an embedding which is summed with the positional encoding. The output of this summation is the input to the encoder layers. The output of the encoder is the input to the decoder."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 0,
      "metadata": {
        "colab": {},
        "colab_type": "code",
        "id": "jpEox7gJ8FCI"
      },
      "outputs": [],
      "source": [
        "class Encoder(tf.keras.layers.Layer):\n",
        "  def __init__(self, num_layers, d_model, num_heads, dff, input_vocab_size, \n",
        "               rate=0.1):\n",
        "    super(Encoder, self).__init__()\n",
        "\n",
        "    self.d_model = d_model\n",
        "    self.num_layers = num_layers\n",
        "    \n",
        "    self.embedding = tf.keras.layers.Embedding(input_vocab_size, d_model)\n",
        "    self.pos_encoding = positional_encoding(input_vocab_size, self.d_model)\n",
        "    \n",
        "    \n",
        "    self.enc_layers = [EncoderLayer(d_model, num_heads, dff, rate) \n",
        "                       for _ in range(num_layers)]\n",
        "  \n",
        "    self.dropout = tf.keras.layers.Dropout(rate)\n",
        "        \n",
        "  def call(self, x, training, mask):\n",
        "\n",
        "    seq_len = tf.shape(x)[1]\n",
        "    \n",
        "    # adding embedding and position encoding.\n",
        "    x = self.embedding(x)  # (batch_size, input_seq_len, d_model)\n",
        "    x *= tf.math.sqrt(tf.cast(self.d_model, tf.float32))\n",
        "    x += self.pos_encoding[:, :seq_len, :]\n",
        "\n",
        "    x = self.dropout(x, training=training)\n",
        "    \n",
        "    for i in range(self.num_layers):\n",
        "      x = self.enc_layers[i](x, training, mask)\n",
        "    \n",
        "    return x  # (batch_size, input_seq_len, d_model)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 0,
      "metadata": {
        "colab": {
          "height": 35
        },
        "colab_type": "code",
        "id": "8QG9nueFQKXx",
        "outputId": "257cdde7-171c-4e85-fec5-70129ee60faf"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "(64, 62, 512)\n"
          ]
        }
      ],
      "source": [
        "sample_encoder = Encoder(num_layers=2, d_model=512, num_heads=8, \n",
        "                         dff=2048, input_vocab_size=8500)\n",
        "\n",
        "sample_encoder_output = sample_encoder(tf.random.uniform((64, 62)), \n",
        "                                       training=False, mask=None)\n",
        "\n",
        "print (sample_encoder_output.shape)  # (batch_size, input_seq_len, d_model)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "p-uO6ls8m2O5"
      },
      "source": [
        "### Decoder"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "ZtT7PKzrXkNr"
      },
      "source": [
        " The `Decoder` consists of:\n",
        "1.   Output Embedding\n",
        "2.   Positional Encoding\n",
        "3.   N decoder layers\n",
        "\n",
        "The target is put through an embedding which is summed with the positional encoding. The output of this summation is the input to the decoder layers. The output of the decoder is the input to the final linear layer."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 0,
      "metadata": {
        "colab": {},
        "colab_type": "code",
        "id": "d5_d5-PLQXwY"
      },
      "outputs": [],
      "source": [
        "class Decoder(tf.keras.layers.Layer):\n",
        "  def __init__(self, num_layers, d_model, num_heads, dff, target_vocab_size, \n",
        "               rate=0.1):\n",
        "    super(Decoder, self).__init__()\n",
        "\n",
        "    self.d_model = d_model\n",
        "    self.num_layers = num_layers\n",
        "    \n",
        "    self.embedding = tf.keras.layers.Embedding(target_vocab_size, d_model)\n",
        "    self.pos_encoding = positional_encoding(target_vocab_size, self.d_model)\n",
        "    \n",
        "    self.dec_layers = [DecoderLayer(d_model, num_heads, dff, rate) \n",
        "                       for _ in range(num_layers)]\n",
        "    self.dropout = tf.keras.layers.Dropout(rate)\n",
        "    \n",
        "  def call(self, x, enc_output, training, \n",
        "           look_ahead_mask, padding_mask):\n",
        "\n",
        "    seq_len = tf.shape(x)[1]\n",
        "    attention_weights = {}\n",
        "    \n",
        "    x = self.embedding(x)  # (batch_size, target_seq_len, d_model)\n",
        "    x *= tf.math.sqrt(tf.cast(self.d_model, tf.float32))\n",
        "    x += self.pos_encoding[:, :seq_len, :]\n",
        "    \n",
        "    x = self.dropout(x, training=training)\n",
        "\n",
        "    for i in range(self.num_layers):\n",
        "      x, block1, block2 = self.dec_layers[i](x, enc_output, training,\n",
        "                                             look_ahead_mask, padding_mask)\n",
        "      \n",
        "      attention_weights['decoder_layer{}_block1'.format(i+1)] = block1\n",
        "      attention_weights['decoder_layer{}_block2'.format(i+1)] = block2\n",
        "    \n",
        "    # x.shape == (batch_size, target_seq_len, d_model)\n",
        "    return x, attention_weights"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 0,
      "metadata": {
        "colab": {
          "height": 35
        },
        "colab_type": "code",
        "id": "a1jXoAMRZyvu",
        "outputId": "8dac9914-2895-447e-e52d-c7ad3008b863"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "(TensorShape([64, 26, 512]), TensorShape([64, 8, 26, 62]))"
            ]
          },
          "execution_count": 0,
          "metadata": {
            "tags": []
          },
          "output_type": "execute_result"
        }
      ],
      "source": [
        "sample_decoder = Decoder(num_layers=2, d_model=512, num_heads=8, \n",
        "                         dff=2048, target_vocab_size=8000)\n",
        "\n",
        "output, attn = sample_decoder(tf.random.uniform((64, 26)), \n",
        "                              enc_output=sample_encoder_output, \n",
        "                              training=False, look_ahead_mask=None, \n",
        "                              padding_mask=None)\n",
        "\n",
        "output.shape, attn['decoder_layer2_block2'].shape"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "y54xnJnuYgJ7"
      },
      "source": [
        "## Create the Transformer"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "uERO1y54cOKq"
      },
      "source": [
        "Transformer consists of the encoder, decoder and a final linear layer. The output of the decoder is the input to the linear layer and its output is returned."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 0,
      "metadata": {
        "colab": {},
        "colab_type": "code",
        "id": "PED3bIpOYkBu"
      },
      "outputs": [],
      "source": [
        "class Transformer(tf.keras.Model):\n",
        "  def __init__(self, num_layers, d_model, num_heads, dff, input_vocab_size, \n",
        "               target_vocab_size, rate=0.1):\n",
        "    super(Transformer, self).__init__()\n",
        "\n",
        "    self.encoder = Encoder(num_layers, d_model, num_heads, dff, \n",
        "                           input_vocab_size, rate)\n",
        "\n",
        "    self.decoder = Decoder(num_layers, d_model, num_heads, dff, \n",
        "                           target_vocab_size, rate)\n",
        "\n",
        "    self.final_layer = tf.keras.layers.Dense(target_vocab_size)\n",
        "    \n",
        "  def call(self, inp, tar, training, enc_padding_mask, \n",
        "           look_ahead_mask, dec_padding_mask):\n",
        "\n",
        "    enc_output = self.encoder(inp, training, enc_padding_mask)  # (batch_size, inp_seq_len, d_model)\n",
        "    \n",
        "    # dec_output.shape == (batch_size, tar_seq_len, d_model)\n",
        "    dec_output, attention_weights = self.decoder(\n",
        "        tar, enc_output, training, look_ahead_mask, dec_padding_mask)\n",
        "    \n",
        "    final_output = self.final_layer(dec_output)  # (batch_size, tar_seq_len, target_vocab_size)\n",
        "    \n",
        "    return final_output, attention_weights"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 0,
      "metadata": {
        "colab": {
          "height": 35
        },
        "colab_type": "code",
        "id": "tJ4fbQcIkHW1",
        "outputId": "e7d187df-98f5-4aa4-8503-fc04df49638d"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "TensorShape([64, 26, 8000])"
            ]
          },
          "execution_count": 0,
          "metadata": {
            "tags": []
          },
          "output_type": "execute_result"
        }
      ],
      "source": [
        "sample_transformer = Transformer(\n",
        "    num_layers=2, d_model=512, num_heads=8, dff=2048, \n",
        "    input_vocab_size=8500, target_vocab_size=8000)\n",
        "\n",
        "temp_input = tf.random.uniform((64, 62))\n",
        "temp_target = tf.random.uniform((64, 26))\n",
        "\n",
        "fn_out, _ = sample_transformer(temp_input, temp_target, training=False, \n",
        "                               enc_padding_mask=None, \n",
        "                               look_ahead_mask=None,\n",
        "                               dec_padding_mask=None)\n",
        "\n",
        "fn_out.shape  # (batch_size, tar_seq_len, target_vocab_size)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "wsINyf1VEQLC"
      },
      "source": [
        "## Set hyperparameters"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "zVjWCxFNcgbt"
      },
      "source": [
        "To keep this example small and relatively fast, the values for *num_layers, d_model, and dff* have been reduced. \n",
        "\n",
        "The values used in the base model of transformer were; *num_layers=6*, *d_model = 512*, *dff = 2048*. See the [paper](https://arxiv.org/abs/1706.03762) for all the other versions of the transformer.\n",
        "\n",
        "Note: By changing the values below, you can get the model that achieved state of the art on many tasks."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 0,
      "metadata": {
        "colab": {},
        "colab_type": "code",
        "id": "lnJn5SLA2ahP"
      },
      "outputs": [],
      "source": [
        "num_layers = 4\n",
        "d_model = 128\n",
        "dff = 512\n",
        "num_heads = 8\n",
        "\n",
        "input_vocab_size = tokenizer_pt.vocab_size + 2\n",
        "target_vocab_size = tokenizer_en.vocab_size + 2\n",
        "dropout_rate = 0.1"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "xYEGhEOtzn5W"
      },
      "source": [
        "## Optimizer"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "GOmWW--yP3zx"
      },
      "source": [
        "Use the Adam optimizer with a custom learning rate scheduler according to the formula in the [paper](https://arxiv.org/abs/1706.03762).\n",
        "\n",
        "$$\\Large{lrate = d_{model}^{-0.5} * min(step{\\_}num^{-0.5}, step{\\_}num * warmup{\\_}steps^{-1.5})}$$\n"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 0,
      "metadata": {
        "colab": {},
        "colab_type": "code",
        "id": "iYQdOO1axwEI"
      },
      "outputs": [],
      "source": [
        "class CustomSchedule(tf.keras.optimizers.schedules.LearningRateSchedule):\n",
        "  def __init__(self, d_model, warmup_steps=4000):\n",
        "    super(CustomSchedule, self).__init__()\n",
        "    \n",
        "    self.d_model = d_model\n",
        "    self.d_model = tf.cast(self.d_model, tf.float32)\n",
        "\n",
        "    self.warmup_steps = warmup_steps\n",
        "    \n",
        "  def __call__(self, step):\n",
        "    arg1 = tf.math.rsqrt(step)\n",
        "    arg2 = step * (self.warmup_steps ** -1.5)\n",
        "    \n",
        "    return tf.math.rsqrt(self.d_model) * tf.math.minimum(arg1, arg2)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 0,
      "metadata": {
        "colab": {},
        "colab_type": "code",
        "id": "7r4scdulztRx"
      },
      "outputs": [],
      "source": [
        "learning_rate = CustomSchedule(d_model)\n",
        "\n",
        "optimizer = tf.keras.optimizers.Adam(learning_rate, beta_1=0.9, beta_2=0.98, \n",
        "                                     epsilon=1e-9)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 0,
      "metadata": {
        "colab": {
          "height": 315
        },
        "colab_type": "code",
        "id": "f33ZCgvHpPdG",
        "outputId": "96ffd092-2752-456f-fe5c-1fcd7f12acdb"
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "\u003cmatplotlib.text.Text at 0x7fa7c8353590\u003e"
            ]
          },
          "execution_count": 0,
          "metadata": {
            "tags": []
          },
          "output_type": "execute_result"
        },
        {
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAbEAAAEYCAYAAADSw8iXAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzs3Xl8VPW9//HXzGTfMyHJzCRhFQghIIvgghZljRAJkYqx\noLTKxdsCcuvVitciS10q/d1bKbm01V41VtSCVZtUwSKoCKIgIDvKFgjJzCRkEoZkQpaZ8/sjZGQM\nIdtsCZ/n48FDcr5nzvnMcTJvzvd8z/eoFEVREEIIIbogta8LEEIIITpKQkwIIUSXJSEmhBCiy5IQ\nE0II0WVJiAkhhOiyJMTawGq1snr1aqxWq69LaVFXqBGkTneTOt1L6nQfb9Xo1RArLCwkJyeHjIwM\ncnJyOHPmTLN1HA4Hy5cvZ+LEiUyePJn169e3qW379u3MmDGDIUOGsHLlyivu/+TJkwwbNqzF9pZY\nrVZyc3P9/gPj7zWC1OluUqd7SZ3u460aAzy69R9YunQps2fPJjMzk/z8fJYsWUJeXp7LOvn5+RQV\nFbFp0yYsFgvZ2dmMGTMGg8Fw1baePXvy7LPP8tFHH1FbW9ts3w6Hg6VLlzJhwgRvvV0hhBAe5rUz\nMYvFwpEjR5g6dSoAmZmZHD58mIqKCpf1NmzYwMyZMwHQarVMmDCBjRs3ttqWkpJCamoqGo3mivt/\n6aWXGDduHL179/bE2xNCCOEDXgsxo9FIYmIiKpWqccdqNQkJCZhMJpf1SkpKMBgMzp/1ej1Go7HV\ntqs5evQo27dv56c//akb3okQQgh/4dXuRF9oaGjg6aef5vnnn3cG6NVYrdZmfbgmk4kRI0a0eJbn\nDzQaDUlJSX5dI0id7iZ1upfU6T4ajYYRI0Y0O1EBiIqKIioqyi378VqI6fV6zGYziqKgUqlwOByU\nlpai0+lc1jMYDJSUlJCeng40nsElJSW12taSsrIyioqKmDdvHoqicOHCBQCqqqpYsWJFs/Xz8vLI\nzc11WTZixAjeeuutjr1xL9Hr9WzZssXXZbRK6nQvqdO9pE730ev1vPXWW9x3333s2bPHpW3BggUs\nXLjQLfvxWohptVpSU1MpKChg2rRpFBQUkJaWRmxsrMt6GRkZrFu3jokTJ1JRUcHmzZt54403Wm27\n3OVzGuv1enbs2OH8OTc3F5vNxq9+9asr1jlnzhyys7NdljX9a6eiohqHw/vzJSuKwhN/anwPD0xO\nJb2v9orrxcVFUF5e5c3SOkTqdC+p072kTvdQq1XExobzP//zP9jtdpc2d52FgZe7E5ctW8bixYtZ\ns2YN0dHRzqHu8+bNY9GiRQwePJisrCz27dvHpEmTUKlUzJ8/n+TkZICrtu3evZtHH32U6upqFEVh\nw4YNPPvss4wZM6ZdNV7tNNfhUHwSYlZbHaUVNQBsP2AkrXdsi+v6or6OkDrdS+p0L6nTffR6vUe3\nr5JHsbRdeXmVTz40x85W8vwbe4gKC6TBrvDiI7cSoGk+Jic+PpKysgter6+9pE73kjrdS+p0D7Va\nRVxchOf34/E9iE4zldsAmHJTL2y1DXx7ptLHFQkhhH+QEOsCTBU2NGoVPxpmIChQzZ7vynxdkhBC\n+AUJsS7AVG4jITaUkKAAhvSJY8+xMhzSCyyEEBJiXYG5ogadNgyAEQPiOV9Vx4ni8z6uSgghfE9C\nzM85HAqlFTZniA3r34MAjZqdh0t9XJkQQviehJifO2e9SINdcYZYaHAAw66LY9dRM3aHw8fVCSGE\nb0mI+bmmkYmJl0IM4MY0HVZbPUdOV7T0MiGEuCZIiPk5s6UxxHRx34fY0H5aQoMD+OqQ2VdlCSGE\nX5AQ83Mmi42w4AAiQwOdywIDNIwcEM/u78qoq7df5dVCCNG9SYj5OZPFhi4urNkM/DcOTuRinZ39\nJ8p9VJkQQviehJifM1lsJMaGNVs+qGcs0eFB7DjU/DEHQghxrZAQ82O1dXYqLtS6XA9rolaruHmw\njv0nyjlfXeeD6oQQwvckxPyYueLSoA5t8xADuO16PXaHwo6DcjYmhLg2SYj5MZPl6iGmjwvnuqRo\nPt9fgjyMQAhxLZIQ82NNw+sTYkNbXOe2oXqM5TZOFFu9VZYQQvgNCTE/ZrLY0EYFExyoaXGdUYMS\nCA7SsHV/iRcrE0II/yAh5sdMlpoWuxKbhAQFMDo1gV1HSrFdrPdSZUII4R8kxPyUoiiNw+tbCTGA\nHw0zUFtv59M9Z71QmRBC+A8JMT91wVZPTW1Dq2diAH31UfTWRfLPbadkgIcQ4poiIeanWhuZeDmV\nSsX4kckUmS/IpMBCiGuKhJifak+IAYwelEBUeBCbd0uXohDi2iEh5qdMFhsBGjVxUSFtWj8wQMPk\nm3rxzfFznKus8XB1QgjhHyTE/JTZYiMxNhS1WtX6ypdMuaUPKlRs2VvswcqEEMJ/eDXECgsLycnJ\nISMjg5ycHM6cOdNsHYfDwfLly5k4cSKTJ09m/fr1bWrbvn07M2bMYMiQIaxcudJlm2vWrCEzM5Pp\n06czY8YMtm3b5rk36SZtHZl4uR4xoYwY0IOt35RQU9vgocqEEMJ/BHhzZ0uXLmX27NlkZmaSn5/P\nkiVLyMvLc1knPz+foqIiNm3ahMViITs7mzFjxmAwGK7a1rNnT5599lk++ugjamtrXbZ5/fXX89BD\nDxEcHMzRo0e5//772b59O0FBQd58+21mdzgorahheP/4dr928o09+frbMrbuK2Hy6J4eqE4IIfyH\n187ELBYLR44cYerUqQBkZmZy+PBhKipcR9Nt2LCBmTNnAqDVapkwYQIbN25stS0lJYXU1FQ0muaz\nW4wZM4bg4GAAUlNTAZrt15+cO38Ru0MhUdvydFMt6WeIJrVnDP/aVUSD3eGB6oQQwn947UzMaDSS\nmJjofLijWq0mISEBk8lEbGysc72SkhIMBoPzZ71ej9FobLWtrd577z1SUlJITEy8YrvVasVqdZ2H\nUKPRoNfr27WfzmiaM1GvDe/Q6++8qRe/X7ePHYdM3DbU0PoLhBDCQ4xGI3a76xPoo6KiiIqKcsv2\nvdqd6Gs7d+5k9erVvPrqqy2uk5eXR25ursuypKQktmzZQlxchKdLBODC4VIA0vrHEx0R3K7XxsdH\nckePCN7fdopNX59l+h0D2jU4xFvi4yN9XUKbSJ3uJXW6V1eoc9asWRQXuw42W7BgAQsXLnTL9r0W\nYnq9HrPZjKIoqFQqHA4HpaWl6HQ6l/UMBgMlJSWkp6cDjSmelJTUaltr9u7dyxNPPMEf//hHevXq\n1eJ6c+bMITs722VZUxdleXkVDofnZ8Q4UVRBeEgAdTV1lNW0/YGX8fGRlJVdAGDiDcm8lH+YTTtO\nMWJA+6+tedLldfozqdO9pE738vc61WoVcXERrF279opnYm7bj9u21AqtVktqaioFBQUAFBQUkJaW\n5tKVCJCRkcG6detQFAWLxcLmzZuZNGlSq22X++HUS/v37+fRRx9l1apVzmtiLYmKiiI5Odnljze7\nEqGxO7GtNzm3ZFRqAvExIRRsL5SpqIQQPqPX65t9p3bJEANYtmwZb7zxBhkZGbz55pusWLECgHnz\n5nHo0CEAsrKySE5OZtKkSeTk5DB//nySk5Nbbdu9ezdjx47ltddeY926ddx+++1s374dgBUrVlBb\nW8vSpUuZPn062dnZHDt2zJtvvV1MbggxjVrNXbf04bT5AnuPnXNTZUII4V9Uivwzvc280Z14sa6B\nX/zPVu7+UV8yb+ndrtf+sHvB7nDw65e/IjBAzbIHR6NW+ce1MX/vBmkidbqX1Ole/l5nU3eix/fj\n8T2IdjFbGqeM6uyZGDSejU27tQ9ny6rZ/W1Zp7cnhBD+RkLMz5gr2jfxb2tuHJSIoUc4739+0iuD\nUoQQwpskxPyMqdyGCkiIbf+NzleiVqvIurUPxnIbXx42uWWbQgjhLyTE/IypwoY2KoSgwOYzj3TU\nyIHx9NJF8u7Wk9TV21t/gRBCdBESYn7GVG5D14Hppq5GrVKRM+46LNZaNn1d5NZtCyGEL0mI+RFF\nUTBX2NB1cLqpqxnYM5bh/XvwwY7TWKvbfgO1EEL4MwkxP2KtrqOm1t6hiX/b4se396O+wcE/tp3y\nyPaFEMLbJMT8iOnSxL+6OPeMTPwhfVw4tw9L4rNvSig+V+2RfQghhDdJiPkRZ4jFeibEAKbd2pvQ\nYA1r//WtTEclhOjyJMT8iNlSQ4BGjTY6xGP7iAwL4u6x/Th6ppKvDps9th8hhPAGCTE/YrLYSNSG\nenx6qLHXG+iti+RvW45ju9jg0X0JIYQnSYj5EXdM/NsWarWK+ycPxFpdx/ufn/T4/oQQwlMkxPyE\n3eGgrLLGKyEG0Ecfxe3Dk9i85yynTf47iagQQlyNhJifOFd5EbtD8VqIAdw9ti9RYUG8+uERGuwO\nr+1XCCHcRULMTzSNTEz0YoiFhwRy/+SBnCmt4sMvT3ttv0II4S4SYn7CObzeiyEGMGJAPKMHJVCw\nvZCzpVVe3bcQQnSWhJifMFtsRIQGEhEa6PV9z5o4gLCQAP7vwyPYHdKtKIToOiTE/ETT8HpfiAwL\n4v5JAzltusCHX57xSQ1CCNEREmJ+wlvD61tyQ2oCowcl8I/PT3Gi5LzP6hBCiPaQEPMDNbUNVFbV\n+TTEAB6YPJDYyGBeyj9ETa3cBC2E8H8SYn6gtKIG8P6gjh8KCwnk3+5K49z5i6zd9J1PaxFCiLaQ\nEPMDRkvjjPLeHF7fkgEpMdx1S2++OGjiy0MmX5cjhBBXJSHmB8yWGlRAYqxvBnb80F1jenNdUjR5\nH32LsVwe2SKE8F9eDbHCwkJycnLIyMggJyeHM2eaj4RzOBwsX76ciRMnMnnyZNavX9+mtu3btzNj\nxgyGDBnCypUr27xNf2Cy2IiLDiEwQOPrUgDQqNX8e9ZgAjVqct89wMU6uT4mhPBPXg2xpUuXMnv2\nbDZu3MhPfvITlixZ0myd/Px8ioqK2LRpE2+99Ra5ubmUlJS02tazZ0+effZZ5s6d265t+gNfj0y8\nEm1UCP+eNRiTxcZrG47Ks8eEEH7JayFmsVg4cuQIU6dOBSAzM5PDhw9TUVHhst6GDRuYOXMmAFqt\nlgkTJrBx48ZW21JSUkhNTUWjaX42c7XX+ZqiKJfuEfOvEANI663l7h/1ZeeRUjZ9fdbX5QghRDMB\n3tqR0WgkMTER1aVnZanVahISEjCZTMTGxjrXKykpwWAwOH/W6/UYjcZW266mPa+zWq1YrVaXZRqN\nBr1e34Z32X7nq+uorbP73ZlYkyk39eJkiZX1nxynZ0IEqb1iW3+REEJcYjQasdvtLsuioqKIiopy\ny/a9FmJdRV5eHrm5uS7LkpKS2LJlC3FxEW7fn+l8LQAD+8QRHx/Z6e25Yxs/9MSc0Ty++nPWvH+Q\n/7foRyTFd/44eKJOT5A63UvqdK+uUOesWbMoLi52WbZgwQIWLlzolu17LcT0ej1msxlFUVCpVDgc\nDkpLS9HpdC7rGQwGSkpKSE9PBxpTPCkpqdW2q2nP6+bMmUN2drbLsqYuyvLyKhwO914bOnrqHACh\nGhVlZZ17rld8fGSnt9GS+dnpPJP3NUv//AVPPXBDp+Z49GSd7iR1upfU6V7+XqdarSIuLoK1a9de\n8UzMbftx25ZaodVqSU1NpaCgAICCggLS0tJcuhIBMjIyWLduHYqiYLFY2Lx5M5MmTWq17XI/HITQ\n1tdB48FNTk52+eOprkRonPg3MEBNbFSwx/bhDgkxoSy4ewjl1ousee+APH9MCNEmer2+2Xdqlwwx\ngGXLlvHGG2+QkZHBm2++yYoVKwCYN28ehw4dAiArK4vk5GQmTZpETk4O8+fPJzk5udW23bt3M3bs\nWF577TXWrVvH7bffzvbt21t9na+Zym0kxoaivnSt0J8NSInhZ3cO4uiZSl7f+K2MWBRC+JxKkW+i\nNvNEd+KTL31JSnw4v8ge0ulteat74b2tJyn4opCpN/dixth+7X69v3eDNJE63UvqdC9/r7OpO9HT\nZGCHDzXYHZyrrOGGgfG+LqVdpt/Wh/PVdXyw4zSRYUFMGpXi65KEENcoCTEfOnf+InaH4rfD61ui\nUql4YPJAqmvqeXvzMSJDA7k5Xdf6C4UQws1k7kQfMpXbAN/PXt8RarWKedPSSO0ZwysfHmHf8XO+\nLkkIcQ1qU4gpisK6det44IEHuOuuuwDYtWsXH374oUeL6+5MlsYQ88fZOtoiMEDDwhlDSU6I4H/f\nO8iBk+W+LkkIcY1pU4itWrWKd955h3vvvdc504VOp+Mvf/mLR4vr7kwWGxGhgZ2658rXQoMD+M97\nh2GIC2P13w9w8JQEmRDCe9oUYu+99x5/+tOfmDp1qnPaqOTkZIqKijxaXHdnttjQxXXNs7DLRYQG\n8th9w51BduiUxdclCSGuEW0KMbvdTnh4OIAzxKqrqwkL6/pfwL5kstjQxXaPY9gUZDptGH/4+34J\nMiGEV7QpxMaOHcvzzz9PXV0d0HiNbNWqVdxxxx0eLa47q6lt4Hx1Xbc4E2sSERrIYznDSIwNY9U7\n+9j9bamvSxJCdHNtCrEnn3yS0tJSRo4cyYULFxg+fDglJSU89thjnq6v23IO6ugmZ2JNIsOCeGLW\ncHrpIlnz/kE+3+8/z20TQnQ/bbpPLCIigjVr1lBeXk5xcTF6vZ74+K51g66/MV8Kse50JtYkPCSQ\nx+4dTu57B3j1w6PUXGxg0uievi5LCNENtelMbPr06QDExcUxdOhQZ4DdfffdnqusmzNZbKhUjRPr\ndkfBQRoemTGUG1ITeHvLcd759AQOmeFMCOFmbToTO336dLNliqJw9qw87bejTBYbPaJDCAzovveb\nBwao+fdpg3kjJIAPvzzNufM1PDR1kK/LEkJ0I1cNsV/96lcA1NfXO//epLi4mOuuu85zlXVzZktN\nl73JuT3UahX3Tx5IfGwo6z85gcVay7J5N/u6LCFEN3HVEOvZs+cV/w4wYsQIMjIyPFNVN6coCqYK\nG/1TPPecMn+iUqm488ZexEeH8vI/D/PYH7ay8O4h6OPCfV2aEKKLu2qILViwAIDrr7+e2267zSsF\nXQsqq+qorbN3yTkTO+OG1ARiI4PJfe8Az76+m4ezBjOkb5yvyxJCdGFtuiZ22223UVdXx6lTp6io\nqHB5GOLNN0vXUHt19TkTO6NfUjT/75EfsfzlL3lx3T7uHtuXKTf1ct5EL4QQ7dGmEPv666/5j//4\nD+rq6qiqqiIiIoLq6mp0Oh2bN2/2dI3dTtPwev01GGIAurhwnrp/JK9tPMrfPztJoekCD04ZRGiw\nPBlICNE+bfrWeP7555k7dy4//elPGTVqFDt37iQ3N5fQ0O45PNzTTBYbQQFqYiKDfV2KzwQHaZh3\nVxq9dZGs++Q4xnIb87PT5TqZEKJd2jS+u7CwkAceeMBl2bx583jttdc8UVO3Z7LYSNSGob7Gu9BU\nKhWTR/fksXuHYa2uY8VrX/PFQaOvyxJCdCFtCrHIyEiqqqoAiI+P5/jx41itVmw2m0eL666aQkw0\nGtRby7KfjaKXLpK//PMIf/nnYS7WNfi6LCFEF9CmEJs4cSKfffYZAD/+8Y954IEHuPvuu2WIfQc0\n2B2cq7x4zY1MbI02KoRf3TecrFv7sOOQieWvfc0Z8wVflyWE8HNtuib21FNPOf/+4IMPMnToUKqr\nq2XYfQeUVdbgUBR0Wrme+ENqtYqsW/uQ2jOGP+cf4pnXv+buH/Vj0qgU1Opru+tVCHFlHZrz6IYb\nbmDs2LFs3brV3fV0e03D63VaGcDQkoE9Y1n+4GiG9I1j3SfHeeHNPZRWSNe1EKK5VkOssLCQjz76\niKNHjzqXbd68mbvvvpsnn3yyXTsrLCwkJyeHjIwMcnJyOHPmTLN1HA4Hy5cvZ+LEiUyePJn169d3\nus1isfDwww8zbdo0pkyZwooVK3A4HO2q3V2+DzE5E7uayLAgFtw9hH/LTONsWTVPv7KTT/acdblH\nUQghrtqd+O6777JkyRKio6OprKxk8eLFfPnll3z77bc8+OCDzJgxo107W7p0KbNnzyYzM5P8/HyW\nLFlCXl6eyzr5+fkUFRWxadMmLBYL2dnZjBkzBoPB0OG2P/3pT/Tr148///nP2O127rvvPv71r3/5\n5Jqe2WIjKiyQsJBAr++7q1GpVNycrmNgzxhe3XCUv/7rO/Z8V8acO1PpES3/CBBCtHIm9vLLL7Nm\nzRq++OILVq9ezQsvvEDPnj3ZuHEjs2bNIiQkpM07slgsHDlyhKlTpwKQmZnJ4cOHqaiocFlvw4YN\nzJw5EwCtVsuECRPYuHFjp9pUKhXV1dUoisLFixdpaGggMTGxzbW7k6lcRia2lzYqhEdnXs8Dkwdy\nvNjKkr/s5KOdZ7D76GxaCOE/rnomVlpaytixYwEYN24carWa//zP/yQwsP1nEUajkcTEROf0Qmq1\nmoSEBEwmE7Gxsc71SkpKMBgMzp/1ej1Go7FTbb/4xS9YuHAht956KzU1NcyePZvhw4dfsU6r1YrV\nanVZptFo0OvdM1mvqaKG6/vJfIHtpVKpuH14Eul9taz913f8bctxvjxkZs6dA+mti/J1eUKIFhiN\nRux2u8uyqKgooqLc83t71RC7/PqDSqUiNDS0QwHmaxs2bCA1NZXXX3+dqqoq5s6dy7/+9S8mTZrU\nbN28vDxyc3NdliUlJbFlyxbi4iI6VUd1TT3W6jr6pcQSHx/ZqW21xFPbdbeO1hkfH8lv+sXzxX4j\nL72/n2fyvibztr7MzvDMtFXd/Xh6m9TpXl2hzlmzZlFcXOyybMGCBSxcuNAt27/qb31NTQ233367\n8+cLFy64/Azw6aeftmlHer0es9mMoiioVCocDgelpaXodDqX9QwGAyUlJaSnpwONKZ6UlNSptrVr\n1/Lcc88BEBERwfjx4/nqq6+uGGJz5swhOzvbZZlGowGgvLwKh6PjAwtOGRvP8CKDNZSVuf8eqPj4\nSI9s193cUecAQyQrHryRv392goKtJ9m65yz33H4dNw1OdNtkwtfS8fQGqdO9/L1OtVpFXFwEa9eu\nveKZmLtcNcR+OOiiM7RaLampqRQUFDBt2jQKCgpIS0tz6UoEyMjIYN26dUycOJGKigo2b97MG2+8\n0aG2tWvXApCcnMznn3/OkCFDqKurY8eOHVcMMHDvae4PXcuz13tCWEgA908eyC1DdLy56Tte/udh\ntuw9y08mDKCPXroYhfAH7roU05Krhtjo0aPdurNly5axePFi1qxZQ3R0NCtXrgQa52FctGgRgwcP\nJisri3379jFp0iRUKhXz588nOTkZoN1tTWdi//Vf/8XSpUuZNm0aDoeDm266yTkIxJtM5TZUKoiP\nkZF17tTPEM1TD9zA9gNG/v7ZSZ7J+5oxQ/XMGNuP6PAgX5cnhPAglSI33rRZZ7sT//SPgxQaL/Db\nf/fMM9j8vXuhiSfrrKltoGB7IZu+LiIwQM2Um3ox8YYUgoM07d6WHE/3kjrdy9/rbOpO9DR5gJMX\nyfB6zwsNDmDmuOv40TAD6z85zrtbT7J591mybu3DrUP1BGg6NEmNEMJPyW+0lyiKgqnCJhP/eolO\nG8bCGUN5cvYI4mNDef2jb1nyfzv5+mipzPohRDciIeYlFRdqqat3yHRTXtY/OYYnZ43gkRlD0ahV\nrHn/IM+8vpsDJ8slzIToBtrUnfj4449fcdhyUFAQOp2OCRMmkJqa6vbiuhOzc85EORPzNpVKxbD+\nPRjaL47tB4zkbz/F79fto48+iqxb+zCkr9Ztw/KFEN7V5odibt68GUVR0Ol0KIrCli1bUKvVnDhx\ngnvvvZf333/f07V2aTK83vfUahW3XW/g+YdvZk7GQKzVdby4fh/PvP41+0+ckzMzIbqgNp2JFRYW\n8tJLLzFy5Ejnsr179/KHP/yBV199la1bt/Lcc88xffp0jxXa1ZksNQQFqomNDPZ1Kde8AI2ascOS\nGDNEzxcHTfzzi0JeXL+f3rpIptzUixED4uX5ZUJ0EW0KsX379nH99de7LEtPT2f//v0A3HbbbZjN\nZvdX142YLDZ0sWHSbeVHAjRqfnS9gVvSdXxx0MSHO06z5v2DJMSGkjG6J9Pu6O/rEoUQrWhTd+Kg\nQYP4/e9/T21tLQC1tbWsWrXKeR3s7NmzREdHe67KbsBssaGLk65Ef9QUZs/Nu4lfTE8nPCSA1z/6\nlrnPbKLgi0Kqaup9XaIQogVtOhP77W9/y2OPPcbIkSOJjo7m/PnzpKen87vf/Q6AyspKli5d6tFC\nu7L6Bgdl52u4Mc03j38RbaNWq7ghNYGRA+P59kwlm/cW897Wk3y44zS3DtUzfmSyDMwRws+0KcSS\nk5N5++23MRqNlJaWEh8f7/LYkyFDhniswO6grLIGRUHOxLoIlUpFaq9YbruhJ3sOGdn41Rk+3VvM\n5t1nSe+jZfzIZIb0i0MtXcNC+Fy7ZuwICgpCq9Vit9spKioCICUlxSOFdScmGV7fZaUkRPBvd6Ux\nc9x1fPZNMZ/sLWbVO/tJiAll3Igkbh2ql6d0C+FDbQqxrVu38tRTT1FWVuayXKVSceTIEY8U1p00\n3SOWGCsh1lVFhwcxbUwfptzUi93flrF591ne3nKc9z4/xY1piYwdZqC3LlIG7gjhZW0KsRUrVvCL\nX/yC7OxsQkJCPF1Tt2O02IgKDyIsRKaq7OoCNGpuTEvkxrRETpsu8PHuIr48ZGLrvhJSEiL40fUG\nbh6cKGdnQnhJm75VrVYrOTk58q/MDjJbbOhiZbqp7qaXLpKHpqZx3/j+fHXYzGf7Sli76TvWfXKc\nGwYmMHaYgf7J0fJ7I4QHtSnEZsyYwd///nd+/OMfe7qebslssTGsfw9flyE8JCwkkDtGJHPHiGQK\nTVa27jPy5SETOw6Z0GnDuCVdx02DE+kRLf+QEcLd2nyz81//+ldefvllevRw/TJuenqyuDLbxXqs\ntnqZbuoa0VsXRW9dFPfecR27jpby+f4S3t16kne3nmRgSgw3p+u4YWCCdC0L4SZt+k265557uOee\nezxdS7cbV185AAAgAElEQVRkstQAMjLxWhMcpOHWoXpuHaqnrLKGLw+Z+OKQmdc2HOWNf33HsP49\nuGWwjvS+WnnGmRCd0KYQy87O9nQd3ZbJUg1IiF3L4mNCuWtMHzJv6c0p4wV2HDTx1REzXx8tJSI0\nkBEDejAqNZHUXjFo1BJoQrRHiyH2/vvvOyf0feedd1rcgFwnuzqTpQa1SkV8jFwPudapVCr6GqLo\na4ji3vHXcfCkha+OmPnqSClb9xkvBVo8owYlkNpTAk2ItmgxxD744ANniP3jH/+44joqlUpCrBUm\ni40eMSHSZSRcBGjUDOvfg2H9e1BXb+fASQu7jpr56rCZrftKiAgNZOTAeEalJjBQAk2IFrUYYi+/\n/LLz73/961+9Ukx3ZLbYpCtRXFVQoIaRA+MZOTD+UqCVs+toKV8eMvPZNyWEhwQwtF8PhvfvQXpf\nLSFBMihEiCbt+m0oLy/HZrO5LJNpp1rmUBTMFhuDesX6uhTRRTQGWgIjByZQW2/nwIly9h47x/4T\n59hxyESARk1a71iGXdd4FhcTIc+nE9c2mXbKgyov1FLX4JAzMdEhwYEabkhN4IbUBOwOB8eKzrP3\n2Dn2Hitj/4lyXv/oW/roo7h1WBL9dBGkJETIjdXimuPVaacKCwtZvHgxlZWVxMTEsHLlSnr27Omy\njsPh4De/+Q3btm1DrVYzd+5c5/D+jrYBfPjhh/zxj38EGsP3tddeQ6vVdvi9tIWxac5ECTHRSRq1\nmtResaT2iiVn/HUUl1Wz9/g5vjlWxl83NP5DMjoiiCF94kjvq2VwHy3hMvWVuAZ4ddqppUuXMnv2\nbDIzM8nPz2fJkiXk5eW5rJOfn09RURGbNm3CYrGQnZ3NmDFjMBgMHW47cOAAa9as4fXXX0er1VJV\nVUVQUFCn3ktbmGX2euEBKpWK5IQIkhMiuOuW3miCA/l012kOnrSw57syth0wolap6JsUxZC+cQzt\nG0dKYoQ8OkZ0S20a8tQ07VRnWCwWjhw5wtSpUwHIzMzk8OHDVFRUuKy3YcMGZs6cCYBWq2XChAls\n3LixU215eXk8+OCDzjOviIgIr4SYqdxGcJCGmAjP70tcu7RRIdw21MDPp6ezatGtPDl7BFNu7kV9\ng4P3tp5k+Wu7eDR3Oy8XHGb7ASMW60VflyyE23ht2imj0UhiYqLzbE6tVpOQkIDJZCI29vuBDyUl\nJS4P3NTr9RiNxk61nThxguTkZGbPno3NZmPixIn8/Oc/v2KdVqsVq9Xqskyj0aDX69v0Pi9nqrCh\niw2T6xTCazRqNf2TY+ifHMPdP+qLtbqOg6fKOXDSwsFT5ew4ZAIau7jTesUy6FIXZUSodD0KzzAa\njdjtdpdlUVFRREVFuWX718S0Uw0NDXz33Xe89tpr1NbWMnfuXAwGA1lZWc3WzcvLIzc312VZUlIS\nW7ZsIS4uol37Lau8yMCescTHR3aq/vbw5r46Q+p0r5bqjI+Hfr3jyLoDHA6F0yYr+46dY9+xMr48\nbOKTvcWoVNDHEM31/eO5vn8PBveJIyTYM8P4u/rx9Dddoc5Zs2ZRXFzssmzBggUsXLjQLdtv9ZNq\nt9s5c+YMP//5zzvVBafX6zGbzSiKgkqlwuFwUFpaik6nc1nPYDBQUlJCeno60JjiSUlJnWpLSkpi\n8uTJBAQEEBAQwPjx4zlw4MAVQ2zOnDnNptnSaDQAlJdX4XAobXq/9Q0OSi02bhyUQFnZhTa9prPi\n4yO9tq/OkDrdqz11RgSqGZOWwJi0BBrsDgqNFzh82sKRwgoKPj/Be58eR6NW0UsXyYCUGAakxNA/\nOdotg0S64/H0JX+vU61WERcXwdq1a694JuYurYaYRqPhzTff7HRqarVaUlNTKSgoYNq0aRQUFJCW\nlubSlQiQkZHBunXrmDhxIhUVFWzevJk33nijU22ZmZls3bqVrKws6uvr2bFjBxkZGVes012nuaWV\nNSjIoA7hvwI0aq5Ljua65GimjelDbb2d42fPc/RMBd8VVfLx10Vs/OoMKiApPoKBKTH0T4lmQEqM\n3J8m2qwjl2Lao019BtOnT+ett95i1qxZndrZsmXLWLx4MWvWrCE6OpqVK1cCMG/ePBYtWsTgwYPJ\nyspi3759TJo0CZVKxfz580lOTgbocNvUqVM5ePAgU6ZMQaPRcOutt3q8e9RULsPrRdcSHKhhcJ/G\n4fkAdfV2ThmtfFdUyXdFlWw7YGTznrMAJMSGMiAlhuuSoumXFI0+LkxGPwqfUCmK0mr/2H333cf+\n/ftJTExEp9O5DFS4lp4n1p7uxA+/PM07n57gf3/5I0I9dH3hh/y9e6GJ1Ole3qqzwe6gqLSKb880\nhtqxs5VUX2wAIDQ4gL6GKPoZouiXFE1fQ1SzLkg5nu7l73U2dSd6Wpu+XWfOnOkcvi7axlRuIzo8\nyGsBJoSnBWjU9NFH0UcfRcaNPZ3Tqp0otnKy5DzHi60UfFFI0z+L9XFh9DNE0y8pin6GaLRe+EIT\n1x55npiHmCpk4l/RvalVKvRx4ejjwrl1aON1j5raBgqNVo6XWDlRfJ5vjp9j24HGW11CgwPolRhB\nb30UvXWR9NZHER8dIregiE5p82nCuXPn2L9/PxUVFVzeAymPYrkyU7mNEQPifV2GEF4VGhzAoN5a\nBvVuvK6mKAqlFTWcKDlPsaWGo6fK+fjrIhrsjd8h4SEB9NJF0lvXFGyRxEVJsIm2a1OIffzxxzz+\n+OP06tWL48ePc91113Hs2DFGjBghIXYFVTX1VNXUy5mYuOapVCoStWEkasOc13Aa7A6Ky6o5ZbJS\naLzAadMFPtp5Bvul680RoYH01jcGW6/ESHomRtBDzthEC9oUYi+++CLPPfccd955J6NGjeL999/n\n73//O8ePH/d0fV2SzJkoRMsCNGp66SLppYuEYY3L6hvsnC2rptBo5ZTpAoXGC3x46jSOS70+ocEa\nUuIjSEmIJCWxccb+pB7hBAVqfPhOhD9oU4iVlJRw5513uixrmmD3iSee8EhhXZmpKcTiJMSEaIvA\nAI1z0Mgdl5bV1dspKquiqPTSH3MV2w4aqd3TeOOsSgX6uHBSEhpDreel/0bLPWzXlDaFWFxcHOfO\nnaNHjx4kJSWxd+9eYmNjcTgcnq6vSzJZbGjUKnpEd/yxNUJc64ICNY2jGw3RzmUOReFcZQ1nzN+H\n2/GzlXx12OxcJyoskJTESJJ6hJPUIxxDfDiGuHAZKdxNtXnuxN27dzN58mR++tOf8sADD6BWq/nZ\nz37m6fq6JLPFRo+YUAI0bXpIgBCijdQqFQmxYSTEhnFDaoJzefXFeoouC7YzpRf4ZG8l9Q3f/0M7\nLiqEpPhwDE3h1qMx3IKDpEuyK2tTiM2bN8/59+nTpzN69Ghqamro16+fxwrrykwWG7rYUF+XIcQ1\nIzwk0PnQ0CYOh0LZ+RqKy6opPldNyblqisuqOVxocY6OVAE9YkJI6hHhDLfB/e0EqxtnMBH+r83n\n1/X19ezbt4/S0lKmTJmCzWbDZrMRFibXfS7nUBTMFTXOqXuEEL6hVqtIjA0jMTbM5XYXu8NBaUVj\nuJWc+z7gDpwsd46QBIiLCkYXF45OG4Y+LuzSf8OJiQiSkZJ+pE0h9u233zpnsTebzUyZMoVdu3bx\n3nvv8eKLL3q6xi7FYr1IfYND5kwUwk9p1GrnTdqXa7A7MFtsVNU7+PZUOSaLDWO5jW0HjNTWfT8L\ne3CQpjHQtGHoLgu3xNhQGS3pA20KsWXLlvHII48wffp0Ro0aBcCoUaP49a9/7dHiuiKzpQYAvYSY\nEF1KgEZNUnwE8fGRDDR8/yQLRVGorKrDVF6N0WLDVG7DaLFx7Ox5vrxsQImKxqdsJ2pDSYwNIyE2\n9NKfMBJiQggMkIDzhDaF2PHjx53P3mo6jQ4LC6O2ttZzlXVRTcPr5UxMiO5BpVIRGxlMbGSwcyaS\nJrX1dswWG6bLwq20wsZXh83Yahu+3wYQGxVMQkxjqCW6BFyoDC7phDaFWFJSEgcPHmTIkCHOZfv3\n76dnz54eK6yrMllsBAdpiA7v+ANEhRBdQ3Cghp6JkfRMbP6E5aqaekoraiitsFFaUYO5oobSSht7\nviujqqbeZd2YiKBLoy5DSYwNJT4mlB7RofSICSEyNFCuwV1Fm0Js0aJFPPzww+Tk5FBfX8+f//xn\n3n77bX7zm994ur4ux2xpnPhXPnRCXNsiQgOJCA2kr6H5Q3ZtF+spraz5PtwuBd2BE+Vsq65zWTc4\nUEOP6JDGPzGhxF/674B6BxqH45q//61N7/6OO+7g5ZdfZv369YwaNYri4mJWr15Nenq6p+vrckwW\nG/2SoltfUQhxzQoLCaS3LpDeuuYBV1PbQPn5i5Sdr+Fc5ff/PXe+hqNFlS6DTKBxEuWms7b4S//t\nER1KfEwI2qiQbn+rQJsjfPDgwQwePNj5s91uZ9WqVSxatMgjhXVF9Q12ys9fZMwQzz6OWwjRfYUG\nB5CcEEFyQvPnrymKQlVNPefOX6ROgZNnKig7f5FzlTWcLatm3/FzznvgmkSEBhIXFYI2KvjSf0OI\ni/7+56jwoC79VO4On4fa7Xb+9Kc/SYhdxlxRgwIkauVGZyGE+6lUKiLDgogMC2o2ihIa71M9X1VH\nWWUN587XUG6txWK9SLn1IuaKGg6frmh2JqdRq1wCThsVQtyln+OiQ9BGhvj1wJNOdaZe/lwx8f3s\n9XpteCtrCiGE+6kvG0k5ICWmWbuiKNgudVdarLWUWy86Q85ireXI6Qoqq2r54Vd7eEgA2qgQYiOD\n0UYGE3NpH9rIEOf+fHVtrlN7lcELrpqG1yfIlFNCCD+kUqkIDwkkPCTwiiMqofGm78qq2h+EXC0V\n1otUXKjllNHKBVt9s9eFBGmcIRcbGUJKYgQ/uTPN02/p6iG2Y8eOFtvq65u/iWudyWIjJiLomh8t\nJITougI06saBItEt/2O8vsFORVVdY7BV1VJxoZYK66X/VtVSUmjhWHGl70PsqaeeuuqL9XoZwHA5\n06Xh9UII0Z0FBmgab9yOuVqvk3cuN101xLZs2eKVIroLs6WGGwbGt76iEEJ0c2q1dx5F5dUHXhUW\nFpKTk0NGRgY5OTmcOXOm2ToOh4Ply5czceJEJk+ezPr16zvd1uTkyZMMGzaMlStXuv29VdXUU1VT\nL9NNCSGEF3n14s3SpUuZPXs2mZmZ5Ofns2TJEvLy8lzWyc/Pp6ioiE2bNmGxWMjOzmbMmDEYDIYO\nt0FjyC1dupQJEyZ45L01DeqQ7kQhhPAer52JWSwWjhw5wtSpUwHIzMzk8OHDVFRUuKy3YcMGZs6c\nCYBWq2XChAls3LixU20AL730EuPGjaN3794eeX+mcgkxIYTwNq+diRmNRhITE53D8tVqNQkJCZhM\nJmJjv38aa0lJifPsCRoHjxiNxk61HT16lO3bt/P666/zv//7v1et02q1YrVaXZZpNJpWB7GYK2xo\n1Cp6xIRcdT0hhLiWGI1G7HbXG6yjoqKIimo+5VZHdPux4A0NDTz99NM8//zzbbqvLS8vj9zcXJdl\nSUlJbNmyhbi45tPANLFU1TU+BTbRt/Mmxsdf+d4PfyN1upfU6V5Sp/vMmjWL4uJil2ULFixg4cKF\nbtm+10JMr9djNptRFAWVSoXD4aC0tBSdTueynsFgoKSkxDm5sNFoJCkpqcNtZWVlFBUVMW/ePBRF\n4cKFCwBUVVWxYsWKZnXOmTOH7Oxsl2UaTeOUK+XlVTgcVx42esZkJT46lLKyCx06Pu4QHx/p0/23\nldTpXlKne0md7qFWq4iLi2Dt2rVXPBNzF6+FmFarJTU1lYKCAqZNm0ZBQQFpaWkuXYkAGRkZrFu3\njokTJ1JRUcHmzZt54403Otym1+tdbtrOzc3FZrPxq1/96op1duQ016EomC01DOkT167XCSFEd+fp\n+4m92p24bNkyFi9ezJo1a4iOjnYOdZ83bx6LFi1i8ODBZGVlsW/fPiZNmoRKpWL+/PkkJycDdLjN\n0yznL9Jgd8jEv0II4WUqRWbxbbOWuhMPnirnf/62jyd+MpyBPWOv8Erv8PfuhSZSp3tJne4ldbpH\nU3eix/fj8T1cA2R4vRBC+IaEmBuYLTWEBmuICg/ydSlCCHFNkRBzA5OlmsTYMHk0jRBCeJmEmBuY\nLDXo4qQrUQghvE1CrJPq6u1YrBfRxUqICSGEt0mIdVJpRQ0KyJmYEEL4gIRYJzXNXp8oZ2JCCOF1\nEmKd5AwxudFZCCG8TkKsk0wWG7GRwYQEdfu5lIUQwu9IiHWS2WKTm5yFEMJHJMQ6QVEUTBYbiRJi\nQgjhExJinVBVU0/1xQZ0sXI9TAghfEFCrBOaBnXI8HohhPANCbFO+H5kooSYEEL4goRYJ5gtNWjU\nKnpEh/i6FCGEuCZJiHWCyWIjITYUjVoOoxBC+IJ8+3aCDK8XQgjfkhDrIIdDwVxRI9fDhBDChyTE\nOqjcepEGu0POxIQQwockxDrIObxeQkwIIXxGQqyDJMSEEML3JMQ6yGSxERocQGRYoK9LEUKIa5aE\nWAc1jUxUqVS+LkUIIa5ZXg2xwsJCcnJyyMjIICcnhzNnzjRbx+FwsHz5ciZOnMjkyZNZv359p9vW\nrFlDZmYm06dPZ8aMGWzbtq3T78VksaGTZ4gJIYRPefUhWEuXLmX27NlkZmaSn5/PkiVLyMvLc1kn\nPz+foqIiNm3ahMViITs7mzFjxmAwGDrcdv311/PQQw8RHBzM0aNHuf/++9m+fTtBQUEdeh+19XYs\n1lq5HiaEED7mtTMxi8XCkSNHmDp1KgCZmZkcPnyYiooKl/U2bNjAzJkzAdBqtUyYMIGNGzd2qm3M\nmDEEBwcDkJqaCtBsv+1hljkThRDCL3jtTMxoNJKYmOi8hqRWq0lISMBkMhEbG+tcr6SkBIPB4PxZ\nr9djNBo71Xa59957j5SUFBITE69Yp9VqxWq1uizTaDTo9Xrnz+aKGkBGJgohRGuMRiN2u91lWVRU\nFFFRUW7Zvle7E31t586drF69mldffbXFdfLy8sjNzXVZlpSUxJYtW4iLiwDgwjclAAzun0BIsH8d\nwvj4SF+X0CZSp3tJne4ldbrPrFmzKC4udlm2YMECFi5c6Jbte+0bWK/XYzabURQFlUqFw+GgtLQU\nnU7nsp7BYKCkpIT09HSgMcWTkpI61Qawd+9ennjiCf74xz/Sq1evFuucM2cO2dnZLss0Gg0A5eVV\nOBwKJ4oqiY0M5oK1hgudOShuFh8fSVmZP1V0ZVKne0md7iV1uodarSIuLoK1a9de8UzMbftx25Za\nodVqSU1NpaCgAICCggLS0tJcuhIBMjIyWLduHYqiYLFY2Lx5M5MmTepU2/79+3n00UdZtWqV85pY\nS6KiokhOTnb5c3lXIjSNTJSuRCGEaI1er2/2nerOEPNqX9iyZctYvHgxa9asITo6mpUrVwIwb948\nFi1axODBg8nKymLfvn1MmjQJlUrF/PnzSU5OBuhw24oVK6itrWXp0qXOM8GVK1fSv3//dr8HRVEw\nW2zcmHbla2pCCCG8R6UoiuLrIrqK8vIqKqtq+Y8/bOO+8f2ZOCrF1yW58PfuhSZSp3tJne4ldbpH\nU3eix/fj8T10M6ZyGV4vhBD+QkKsnZruEdPFSYgJIYSvSYi1k8liI0CjokdUiK9LEUKIa56EWDuZ\nLDYSYsNQq2XiXyGE8DUJsXYyWWwkxsrEv0II4Q8kxNrB4VAoraiR62FCCOEnJMTaoeLCRewOBV2s\nhJgQQvgDCbF2KDtfC8jIRCGE8BcSYu1QVtk4e73cIyaEEP5BQqwdzlXWEB4SQGRooK9LEUIIgYRY\nu5RV1pCoDXM+E00IIYRvSYi1Q9n5iyTKoA4hhPAbEmLtcL6qVgZ1CCGEH5EQayd5jpgQQvgPCbF2\nkhATQgj/ISHWTgky5ZQQQvgNCbF2iI0IJjhQ4+syhBBCXCIh1g49YuTxK0II4U8kxNohPka6EoUQ\nwp9IiLVDDwkxIYTwKxJi7ZAQLSEmhBD+REKsHeSamBBC+BevhlhhYSE5OTlkZGSQk5PDmTNnmq3j\ncDhYvnw5EydOZPLkyaxfv96jbe0RExHcodcJIYTwjABv7mzp0qXMnj2bzMxM8vPzWbJkCXl5eS7r\n5OfnU1RUxKZNm7BYLGRnZzNmzBgMBoNH2tpDrVbhcCjuPCRCCCE6wWtnYhaLhSNHjjB16lQAMjMz\nOXz4MBUVFS7rbdiwgZkzZwKg1WqZMGECGzdu9FibEEKIrstrIWY0GklMTHQ+xkStVpOQkIDJZHJZ\nr6SkxOUMSa/XYzQaPdYmhBCi6/Jqd2JXYLVasVqtLss0Gg16vR612v+fI9YVagSp092kTveSOjuv\nqTaj0Yjdbndpi4qKIioqyi378VqI6fV6zGYziqKgUqlwOByUlpai0+lc1jMYDJSUlJCeng40HoCk\npCSPtf1QXl4eubm5LstGjBjBW2+9RWxsuDsOhUfFxUX4uoQ2kTrdS+p0L6nTfR599FH27NnjsmzB\nggUsXLjQPTtQvOj+++9X/vGPfyiKoijvv/++8sADDzRb591331UeeughxeFwKOXl5crYsWOVoqIi\nj7X90Pnz55WioiKXP7t27VJycnKUkpISTxwWtygpKVHuuOMOv65RUaROd5M63UvqdJ+SkhIlJydH\n2bVrV7Pv1PPnz7ttP17tTly2bBmLFy9mzZo1REdHs3LlSgDmzZvHokWLGDx4MFlZWezbt49Jkyah\nUqmYP38+ycnJAB5p+6GWTnP37NnT7JTYn9jtdoqLi/26RpA63U3qdC+p033sdjt79uxBp9O1+H3r\nDl4Nsb59+7Ju3bpmy1966SXn39VqNcuWLbvi6z3RJoQQouuSGTuEEEJ0WRJiQgghuizNMulna5Pg\n4GBuvPFGgoP9d+qprlAjSJ3uJnW6l9TpPt6oUaUoisyjJIQQokuS7kQhhBBdloSYEEKILktCrBVt\neXyMp4wbN44pU6Ywffp0srOz2b59OwDffPMNWVlZZGRk8NBDD2GxWJyv6Whbe7zwwguMHz+e1NRU\njh8/7lx+tWPlibaO1tnScQXvH9vKykrmzZvHnXfeSVZWFo888ohzUmxP1OKJOlNTU8nKynIez2PH\njjlft2XLFu68804mT57Mo48+Sm1tbafbWjN//nxnLbNnz+bo0aOA/30+W6rTnz6fTXJzc11+j/zp\ns+nVGTu6ogceeEApKChQFEVR/vGPf1xxlhFPGTdunHL8+PFmyydOnKjs2bNHURRFWbNmjfLkk092\nuq09du/erZhMJmXcuHHKsWPHnMuvdqw80dbROls6rori/WNbWVmp7Ny50/nzCy+8oDz11FMeq8UT\ndaampio1NTXNXlNdXa2MGTNGOXPmjKIoivLUU08pubm5nWpriwsXLjj//vHHHyvZ2dmKovjf57Ol\nOu+44w6/+XwqiqIcOnRImTt3rnLHHXc4f4/86bMpIXYV5eXlyqhRoxSHw6EoiqLY7XblhhtuUCwW\ni1f2f/mHpsn+/fuVzMxM588Wi0UZNmxYp9rcUd/VjpUn2jpa55V+buIPx/ajjz5Sfvazn3mkFk/U\nqSiKMnDgQMVmszVbZ8OGDcrDDz/s/PnAgQPO/be3berUqR2q87333lNmzJjh15/Ppjp//OMfK4ri\nX5/P2tpa5d5771XOnj3rrMvfPpsyi/1VXO3xMbGxsV6p4bHHHkNRFEaOHMkvf/nLZpMXN9VhtVo7\n3OaO2aSvdqwcDofb2zp7/C8/ro8++igRERE+P7aKovDWW28xfvx4j9Ti7jonTJgAgEql4v7778du\nt3PbbbexcOFCAgMDm+3PYDA4H4HU3rYfPrKpNb/+9a+d3XB/+ctf/Pbz+cM6m/jL5/MPf/gDWVlZ\nLq/3t8+mXBPzY2+99Rbvv/8+77zzDg6HgxUrVlxxPeUqd0l0tK07++FxXb58eYvrevPYrlixgvDw\ncGbPnu21WjpT56xZswD49NNPeeedd3jjjTc4fvw4a9asafc23e2ZZ57hk08+4Ze//CUvvPAC4J+f\n9yvV6S+fz2+++YYDBw5w3333tfo6X342JcSu4vLHxwAtPj7GUxITEwEIDAzkJz/5CXv37sVgMFBc\nXOxcx2KxoFKpiIqKQq/Xd6jNHa52rDzR1hlXOq5N78FXx/aFF17gzJkzvPjiix6rxRN1wvfHMzw8\nnHvuucf52I0f7q+kpAS9Xt+ptvaaNm0aX331ld9/PpvqPH/+vN98Pnfu3MmpU6cYP34848aNw2w2\nM3fuXM6cOeNXn00JsavQarWkpqZSUFAAQEFBAWlpaV7pSqypqaGqqsr58wcffEBaWhqDBw+mtrbW\n+UXx9ttvc+eddwKQnp7eobbOaPoFvtqx8kRbR13puA4aNAjo+PHr7LH9/e9/z+HDh1mzZg0BAQEe\nq8UTdVqtVufIwYaGBj766CPn8bzttts4ePCgc8Te22+/TUZGRqfaWmOz2Vy6Hrds2UJMTAxarZZB\ngwb5zeezpTqDg4P95vM5b948tm7dyubNm9myZQuJiYm88sorPPTQQ3712ZQZO1px8uRJFi9ejNVq\nJTo6mhdeeIHevXt7fL9FRUU88sgjOBwOHA4H/fr149e//jU9evTgm2++YcmSJdTV1ZGcnMzvfvc7\ntFotQIfb2uOZZ55h06ZNlJeXExMTQ2xsLAUFBVc9Vp5o60idf/zjH1m4cOEVj2tnjl9Hj+3x48e5\n66676N27t3NqnpSUFFavXs3evXt5+umn3VqLu+pUqVQkJyfz0EMP8fTTT6NWq2loaGD48OH813/9\nF6GhoUDjl/PKlStRFIVBgwbx29/+lpCQkE61XU15eTm/+MUvqKmpQa1WExMTwxNPPMGgQYP86vN5\npToXL15MeHh4i7/3vvj/frnx48fz5z//meuuu84jdXS0RgkxIYQQXZZ0JwohhOiyJMSEEEJ0WRJi\nQiVntKIAAASXSURBVAghuiwJMSGEEF2WhJgQQoguS0JMCCFElyUhJoQfcjgcDB8+vN1zBgpxrZH7\nxIRwg+HDhzsngq2pqSEoKAi1Wo1KpWLFihVkZmZ6tZ6//e1vvPrqq5SVlREaGkp6ejovvvgiISEh\nPP744/Tq1YsFCxZ4tSYhPEFmsRfCDZrmt4PGmQ2effZZbrrpphbXt9vtaDQaj9SyY8cOcnNz+b//\n+z8GDBjA+fPn+eSTTzyyLyF8TUJMCDdTGp/T57LsxRdf5PTp06jVaj799FOWLFlC7969ef755zl5\n8iQhISFMnjyZJ598Eo1Gg91uZ/DgwWzZsgWDwcDjjz9OdHQ0p0+fZvfu3QwYMID//u//dnl8RZMD\nBw4wYsQIBgwYAEB0dDTTp08H4M0332TDhg2oVCpeeeUVxowZw+rVqzGbzfzmN79h9+7dhIeH87Of\n/cw5S31T7Q6Hg88//5y+ffvy3HPPObcvhC/JNTEhvOTjjz9m2rRp7N69mylTphAQEMBTTz3Fzp07\neeutt9i2bRtvv/22c/2m7skmH3zwAb/85S/ZtWsXer2eVatWXXE/w4YN47PPPiM3N5e9e/dSV1fn\nbPvJT37CnXfeycMPP8yePXtYvXo1DoeDhx9+mKFDh7Jt2zZeffVVXnnlFb788kvn6zZt2sS0adPY\ntWsXkyZNYsGCBTgcDjcfISHaT0JMCC8ZOXIkY8eOBSAoKIj09HSGDh3qnEx35syZ7Nq1y7n+D8/m\nJk+eTFpaGhqNhrvuuosjR45ccT+jR49m1apVHDx4kHnz5nHzzTezcuXKFuv65ptvqK6uZt68eWg0\nGlJSUpgxYwYffvihc53rr7+e8ePHo9FomDt3LlVVVRw4cKAzh0MIt5DuRCG85IfPxDp58iQvvPAC\nhw4doqamBofDwdChQ1t8fdNM5gChoaHYbLYW1x07dqwzMHfs2MEjjzxCv379mDFjRrN1S0pKKCkp\nYfTo0UBjeDocDm688UbnOpc/I0utVpOYmEhpaWkr71gIz5MQE8JHli5dyrBhw1i1ahUhISG88sor\nfPrpp27fz80338zo0aM5duwY0LybUqfT0bt3bz744IMWt3H5UH9FUTCbzSQkJLi9ViHaS7oThfCR\n6upqIiMjCQkJ4cSJE/ztb39zy3Y//vhjNmzYgNVqBRq7C7/++muGDRsGQFxcHGfPnnWuP3z4cAID\nA3n11Vepq6vDbrfz3XffcejQIec6+/fvZ8uWLTQ0NPCXv/yFiIgIhgwZ4pZ6hegMCTEh3OyHZzot\neeKJJ3j33XcZMWIEy5YtY8qUKS1up63bBIiMjOTtt99m0qRJjBw5kieffJKf//znzqcj33PPPRw5\ncoQbb7yRRYsWodFoeOmll9i/fz/jxo3jlltuYenSpVRXVzu3OXHiRPLz8xk9ejQbN24kNzcXtVq+\nPoTvyc3OQoirevHFFzGbzTz//PO+LkWIZuSfUkIIIbosCTEhhBBdlnQnCiGE6LLkTEwIIUSXJSEm\nhBCiy5IQE0II0WVJiAkhhOiyJMTE/2+vDkgAAAAABP1/3Y5ATwiwJTEAtgJYshoTeWVfpgAAAABJ\nRU5ErkJggg==\n",
            "text/plain": [
              "\u003cmatplotlib.figure.Figure at 0x7fa7c5b99f90\u003e"
            ]
          },
          "metadata": {
            "tags": []
          },
          "output_type": "display_data"
        }
      ],
      "source": [
        "temp_learning_rate_schedule = CustomSchedule(d_model)\n",
        "\n",
        "plt.plot(temp_learning_rate_schedule(tf.range(40000, dtype=tf.float32)))\n",
        "plt.ylabel(\"Learning Rate\")\n",
        "plt.xlabel(\"Train Step\")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "YgkDE7hzo8r5"
      },
      "source": [
        "## Loss and metrics"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "oxGJtoDuYIHL"
      },
      "source": [
        "Since the target sequences are padded, it is important to apply a padding mask when calculating the loss."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 0,
      "metadata": {
        "colab": {},
        "colab_type": "code",
        "id": "MlhsJMm0TW_B"
      },
      "outputs": [],
      "source": [
        "loss_object = tf.keras.losses.SparseCategoricalCrossentropy(\n",
        "    from_logits=True, reduction='none')"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 0,
      "metadata": {
        "colab": {},
        "colab_type": "code",
        "id": "67oqVHiT0Eiu"
      },
      "outputs": [],
      "source": [
        "def loss_function(real, pred):\n",
        "  mask = tf.math.logical_not(tf.math.equal(real, 0))\n",
        "  loss_ = loss_object(real, pred)\n",
        "\n",
        "  mask = tf.cast(mask, dtype=loss_.dtype)\n",
        "  loss_ *= mask\n",
        "  \n",
        "  return tf.reduce_mean(loss_)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 0,
      "metadata": {
        "colab": {},
        "colab_type": "code",
        "id": "phlyxMnm-Tpx"
      },
      "outputs": [],
      "source": [
        "train_loss = tf.keras.metrics.Mean(name='train_loss')\n",
        "train_accuracy = tf.keras.metrics.SparseCategoricalAccuracy(\n",
        "    name='train_accuracy')"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "aeHumfr7zmMa"
      },
      "source": [
        "## Training and checkpointing"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 0,
      "metadata": {
        "colab": {},
        "colab_type": "code",
        "id": "UiysUa--4tOU"
      },
      "outputs": [],
      "source": [
        "transformer = Transformer(num_layers, d_model, num_heads, dff,\n",
        "                          input_vocab_size, target_vocab_size, dropout_rate)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 0,
      "metadata": {
        "colab": {},
        "colab_type": "code",
        "id": "ZOJUSB1T8GjM"
      },
      "outputs": [],
      "source": [
        "def create_masks(inp, tar):\n",
        "  # Encoder padding mask\n",
        "  enc_padding_mask = create_padding_mask(inp)\n",
        "  \n",
        "  # Used in the 2nd attention block in the decoder.\n",
        "  # This padding mask is used to mask the encoder outputs.\n",
        "  dec_padding_mask = create_padding_mask(inp)\n",
        "  \n",
        "  # Used in the 1st attention block in the decoder.\n",
        "  # It is used to pad and mask future tokens in the input received by \n",
        "  # the decoder.\n",
        "  look_ahead_mask = create_look_ahead_mask(tf.shape(tar)[1])\n",
        "  dec_target_padding_mask = create_padding_mask(tar)\n",
        "  combined_mask = tf.maximum(dec_target_padding_mask, look_ahead_mask)\n",
        "  \n",
        "  return enc_padding_mask, combined_mask, dec_padding_mask"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "Fzuf06YZp66w"
      },
      "source": [
        "Create the checkpoint path and the checkpoint manager. This will be used to save checkpoints every `n` epochs."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 0,
      "metadata": {
        "colab": {},
        "colab_type": "code",
        "id": "hNhuYfllndLZ"
      },
      "outputs": [],
      "source": [
        "checkpoint_path = \"./checkpoints/train\"\n",
        "\n",
        "ckpt = tf.train.Checkpoint(transformer=transformer,\n",
        "                           optimizer=optimizer)\n",
        "\n",
        "ckpt_manager = tf.train.CheckpointManager(ckpt, checkpoint_path, max_to_keep=5)\n",
        "\n",
        "# if a checkpoint exists, restore the latest checkpoint.\n",
        "if ckpt_manager.latest_checkpoint:\n",
        "  ckpt.restore(ckpt_manager.latest_checkpoint)\n",
        "  print ('Latest checkpoint restored!!')"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "0Di_Yaa1gf9r"
      },
      "source": [
        "The target is divided into tar_inp and tar_real. tar_inp is passed as an input to the decoder. `tar_real` is that same input shifted by 1: At each location in `tar_input`, `tar_real` contains the  next token that should be predicted.\n",
        "\n",
        "For example, `sentence` = \"SOS A lion in the jungle is sleeping EOS\"\n",
        "\n",
        "`tar_inp` =  \"SOS A lion in the jungle is sleeping\"\n",
        "\n",
        "`tar_real` = \"A lion in the jungle is sleeping EOS\"\n",
        "\n",
        "The transformer is an auto-regressive model: it makes predictions one part at a time, and uses its output so far to decide what to do next. \n",
        "\n",
        "During training this example uses teacher-forcing (like in the [text generation tutorial](./text_generation.ipynb)). Teacher forcing is passing the true output to the next time step regardless of what the model predicts at the current time step.\n",
        "\n",
        "As the transformer predicts each word, *self-attention* allows it to look at the previous words in the input sequence to better predict the next word.\n",
        "\n",
        "To prevent the model from peaking at the expected output the model uses a look-ahead mask."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 0,
      "metadata": {
        "colab": {},
        "colab_type": "code",
        "id": "LKpoA6q1sJFj"
      },
      "outputs": [],
      "source": [
        "EPOCHS = 20"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 0,
      "metadata": {
        "colab": {},
        "colab_type": "code",
        "id": "iJwmp9OE29oj"
      },
      "outputs": [],
      "source": [
        "@tf.function\n",
        "def train_step(inp, tar):\n",
        "  tar_inp = tar[:, :-1]\n",
        "  tar_real = tar[:, 1:]\n",
        "  \n",
        "  enc_padding_mask, combined_mask, dec_padding_mask = create_masks(inp, tar_inp)\n",
        "  \n",
        "  with tf.GradientTape() as tape:\n",
        "    predictions, _ = transformer(inp, tar_inp, \n",
        "                                 True, \n",
        "                                 enc_padding_mask, \n",
        "                                 combined_mask, \n",
        "                                 dec_padding_mask)\n",
        "    loss = loss_function(tar_real, predictions)\n",
        "\n",
        "  gradients = tape.gradient(loss, transformer.trainable_variables)    \n",
        "  optimizer.apply_gradients(zip(gradients, transformer.trainable_variables))\n",
        "  \n",
        "  train_loss(loss)\n",
        "  train_accuracy(tar_real, predictions)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "qM2PDWGDJ_8V"
      },
      "source": [
        "Portuguese is used as the input language and English is the target language."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 0,
      "metadata": {
        "colab": {
          "height": 1889
        },
        "colab_type": "code",
        "id": "bbvmaKNiznHZ",
        "outputId": "6671d711-cb76-4e57-f6f0-516b3eb64990"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Epoch 1 Batch 0 Loss 4.4091 Accuracy 0.0000\n",
            "Epoch 1 Batch 500 Loss 2.8389 Accuracy 0.4932\n",
            "Epoch 1 Loss 2.4228 Accuracy 0.5203\n",
            "Time taken for 1 epoch: 319.307568073 secs\n",
            "\n",
            "Epoch 2 Batch 0 Loss 1.3543 Accuracy 0.6036\n",
            "Epoch 2 Batch 500 Loss 1.1689 Accuracy 0.6405\n",
            "Epoch 2 Loss 1.1440 Accuracy 0.6460\n",
            "Time taken for 1 epoch: 251.259027004 secs\n",
            "\n",
            "Epoch 3 Batch 0 Loss 1.1281 Accuracy 0.6554\n",
            "Epoch 3 Batch 500 Loss 1.0244 Accuracy 0.6723\n",
            "Epoch 3 Loss 1.0115 Accuracy 0.6754\n",
            "Time taken for 1 epoch: 70.5513730049 secs\n",
            "\n",
            "Epoch 4 Batch 0 Loss 1.0113 Accuracy 0.6764\n",
            "Epoch 4 Batch 500 Loss 0.9192 Accuracy 0.6975\n",
            "Epoch 4 Loss 0.9037 Accuracy 0.7019\n",
            "Time taken for 1 epoch: 70.917550087 secs\n",
            "\n",
            "Epoch 5 Batch 0 Loss 0.9030 Accuracy 0.7027\n",
            "Epoch 5 Batch 500 Loss 0.8099 Accuracy 0.7260\n",
            "Saving checkpoint for epoch 5 at ./checkpoints/train/ckpt-1\n",
            "Epoch 5 Loss 0.7974 Accuracy 0.7293\n",
            "Time taken for 1 epoch: 73.0342350006 secs\n",
            "\n",
            "Epoch 6 Batch 0 Loss 0.8077 Accuracy 0.7360\n",
            "Epoch 6 Batch 500 Loss 0.7201 Accuracy 0.7475\n",
            "Epoch 6 Loss 0.7084 Accuracy 0.7503\n",
            "Time taken for 1 epoch: 70.6219291687 secs\n",
            "\n",
            "Epoch 7 Batch 0 Loss 0.7275 Accuracy 0.7451\n",
            "Epoch 7 Batch 500 Loss 0.6304 Accuracy 0.7688\n",
            "Epoch 7 Loss 0.6182 Accuracy 0.7719\n",
            "Time taken for 1 epoch: 72.2072319984 secs\n",
            "\n",
            "Epoch 8 Batch 0 Loss 0.6404 Accuracy 0.7730\n",
            "Epoch 8 Batch 500 Loss 0.5517 Accuracy 0.7887\n",
            "Epoch 8 Loss 0.5430 Accuracy 0.7911\n",
            "Time taken for 1 epoch: 70.9613239765 secs\n",
            "\n",
            "Epoch 9 Batch 0 Loss 0.5784 Accuracy 0.7829\n",
            "Epoch 9 Batch 500 Loss 0.4962 Accuracy 0.8035\n",
            "Epoch 9 Loss 0.4900 Accuracy 0.8052\n",
            "Time taken for 1 epoch: 68.5947010517 secs\n",
            "\n",
            "Epoch 10 Batch 0 Loss 0.5201 Accuracy 0.7956\n",
            "Epoch 10 Batch 500 Loss 0.4545 Accuracy 0.8145\n",
            "Saving checkpoint for epoch 10 at ./checkpoints/train/ckpt-2\n",
            "Epoch 10 Loss 0.4493 Accuracy 0.8161\n",
            "Time taken for 1 epoch: 68.6737399101 secs\n",
            "\n",
            "Epoch 11 Batch 0 Loss 0.4883 Accuracy 0.8055\n",
            "Epoch 11 Batch 500 Loss 0.4206 Accuracy 0.8240\n",
            "Epoch 11 Loss 0.4164 Accuracy 0.8251\n",
            "Time taken for 1 epoch: 69.4070420265 secs\n",
            "\n",
            "Epoch 12 Batch 0 Loss 0.4413 Accuracy 0.8195\n",
            "Epoch 12 Batch 500 Loss 0.3937 Accuracy 0.8317\n",
            "Epoch 12 Loss 0.3902 Accuracy 0.8328\n",
            "Time taken for 1 epoch: 70.7441010475 secs\n",
            "\n",
            "Epoch 13 Batch 0 Loss 0.4223 Accuracy 0.8236\n",
            "Epoch 13 Batch 500 Loss 0.3716 Accuracy 0.8380\n",
            "Epoch 13 Loss 0.3685 Accuracy 0.8389\n",
            "Time taken for 1 epoch: 71.3240449429 secs\n",
            "\n",
            "Epoch 14 Batch 0 Loss 0.4037 Accuracy 0.8265\n",
            "Epoch 14 Batch 500 Loss 0.3511 Accuracy 0.8442\n",
            "Epoch 14 Loss 0.3483 Accuracy 0.8450\n",
            "Time taken for 1 epoch: 75.1278469563 secs\n",
            "\n",
            "Epoch 15 Batch 0 Loss 0.3782 Accuracy 0.8331\n",
            "Epoch 15 Batch 500 Loss 0.3339 Accuracy 0.8493\n",
            "Saving checkpoint for epoch 15 at ./checkpoints/train/ckpt-3\n",
            "Epoch 15 Loss 0.3318 Accuracy 0.8499\n",
            "Time taken for 1 epoch: 71.4256718159 secs\n",
            "\n",
            "Epoch 16 Batch 0 Loss 0.3577 Accuracy 0.8409\n",
            "Epoch 16 Batch 500 Loss 0.3195 Accuracy 0.8537\n",
            "Epoch 16 Loss 0.3172 Accuracy 0.8544\n",
            "Time taken for 1 epoch: 70.8179049492 secs\n",
            "\n",
            "Epoch 17 Batch 0 Loss 0.3447 Accuracy 0.8466\n",
            "Epoch 17 Batch 500 Loss 0.3055 Accuracy 0.8579\n",
            "Epoch 17 Loss 0.3033 Accuracy 0.8587\n",
            "Time taken for 1 epoch: 68.7967669964 secs\n",
            "\n",
            "Epoch 18 Batch 0 Loss 0.3385 Accuracy 0.8487\n",
            "Epoch 18 Batch 500 Loss 0.2931 Accuracy 0.8620\n",
            "Epoch 18 Loss 0.2910 Accuracy 0.8626\n",
            "Time taken for 1 epoch: 67.865557909 secs\n",
            "\n",
            "Epoch 19 Batch 0 Loss 0.3198 Accuracy 0.8503\n",
            "Epoch 19 Batch 500 Loss 0.2818 Accuracy 0.8657\n",
            "Epoch 19 Loss 0.2797 Accuracy 0.8665\n",
            "Time taken for 1 epoch: 67.9785480499 secs\n",
            "\n",
            "Epoch 20 Batch 0 Loss 0.3110 Accuracy 0.8557\n",
            "Epoch 20 Batch 500 Loss 0.2726 Accuracy 0.8684\n",
            "Saving checkpoint for epoch 20 at ./checkpoints/train/ckpt-4\n",
            "Epoch 20 Loss 0.2706 Accuracy 0.8692\n",
            "Time taken for 1 epoch: 71.4560930729 secs\n",
            "\n"
          ]
        }
      ],
      "source": [
        "for epoch in range(EPOCHS):\n",
        "  start = time.time()\n",
        "  \n",
        "  train_loss.reset_states()\n",
        "  train_accuracy.reset_states()\n",
        "  \n",
        "  # inp -\u003e portuguese, tar -\u003e english\n",
        "  for (batch, (inp, tar)) in enumerate(train_dataset):\n",
        "    train_step(inp, tar)\n",
        "    \n",
        "    if batch % 500 == 0:\n",
        "      print ('Epoch {} Batch {} Loss {:.4f} Accuracy {:.4f}'.format(\n",
        "          epoch + 1, batch, train_loss.result(), train_accuracy.result()))\n",
        "      \n",
        "  if (epoch + 1) % 5 == 0:\n",
        "    ckpt_save_path = ckpt_manager.save()\n",
        "    print ('Saving checkpoint for epoch {} at {}'.format(epoch+1,\n",
        "                                                         ckpt_save_path))\n",
        "    \n",
        "  print ('Epoch {} Loss {:.4f} Accuracy {:.4f}'.format(epoch + 1, \n",
        "                                                train_loss.result(), \n",
        "                                                train_accuracy.result()))\n",
        "\n",
        "  print ('Time taken for 1 epoch: {} secs\\n'.format(time.time() - start))"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "QfcsSWswSdGV"
      },
      "source": [
        "## Evaluate"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "y6APsFrgImLW"
      },
      "source": [
        "The following steps are used for evaluation:\n",
        "\n",
        "* Encode the input sentence using the Portuguese tokenizer (`tokenizer_pt`). Moreover, add the start and end token so the input is equivalent to what the model is trained with. This is the encoder input.\n",
        "* The decoder input is the `start token == tokenizer_en.vocab_size`.\n",
        "* Calculate the padding masks and the look ahead masks.\n",
        "* The `decoder` then outputs the predictions by looking at the `encoder output` and its own output (self-attention).\n",
        "* Select the last word and calculate the argmax of that.\n",
        "* Concatentate the predicted word to the decoder input as pass it to the decoder.\n",
        "* In this approach, the decoder predicts the next word based on the previous words it predicted.\n",
        "\n",
        "Note: The model used here has less capacity to keep the example relatively faster so the predictions maybe less right. To reproduce the results in the paper, use the entire dataset and base transformer model or transformer XL, by changing the hyperparameters above."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 0,
      "metadata": {
        "colab": {},
        "colab_type": "code",
        "id": "5buvMlnvyrFm"
      },
      "outputs": [],
      "source": [
        "def evaluate(inp_sentence):\n",
        "  start_token = [tokenizer_pt.vocab_size]\n",
        "  end_token = [tokenizer_pt.vocab_size + 1]\n",
        "  \n",
        "  # inp sentence is portuguese, hence adding the start and end token\n",
        "  inp_sentence = start_token + tokenizer_pt.encode(inp_sentence) + end_token\n",
        "  encoder_input = tf.expand_dims(inp_sentence, 0)\n",
        "  \n",
        "  # as the target is english, the first word to the transformer should be the\n",
        "  # english start token.\n",
        "  decoder_input = [tokenizer_en.vocab_size]\n",
        "  output = tf.expand_dims(decoder_input, 0)\n",
        "    \n",
        "  for i in range(MAX_LENGTH):\n",
        "    enc_padding_mask, combined_mask, dec_padding_mask = create_masks(\n",
        "        encoder_input, output)\n",
        "  \n",
        "    # predictions.shape == (batch_size, seq_len, vocab_size)\n",
        "    predictions, attention_weights = transformer(encoder_input, \n",
        "                                                 output,\n",
        "                                                 False,\n",
        "                                                 enc_padding_mask,\n",
        "                                                 combined_mask,\n",
        "                                                 dec_padding_mask)\n",
        "    \n",
        "    # select the last word from the seq_len dimension\n",
        "    predictions = predictions[: ,-1:, :]  # (batch_size, 1, vocab_size)\n",
        "\n",
        "    predicted_id = tf.cast(tf.argmax(predictions, axis=-1), tf.int32)\n",
        "    \n",
        "    # return the result if the predicted_id is equal to the end token\n",
        "    if tf.equal(predicted_id, tokenizer_en.vocab_size+1):\n",
        "      return tf.squeeze(output, axis=0), attention_weights\n",
        "    \n",
        "    # concatentate the predicted_id to the output which is given to the decoder\n",
        "    # as its input.\n",
        "    output = tf.concat([output, predicted_id], axis=-1)\n",
        "\n",
        "  return tf.squeeze(output, axis=0), attention_weights"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 0,
      "metadata": {
        "colab": {},
        "colab_type": "code",
        "id": "CN-BV43FMBej"
      },
      "outputs": [],
      "source": [
        "def plot_attention_weights(attention, sentence, result, layer):\n",
        "  fig = plt.figure(figsize=(16, 8))\n",
        "  \n",
        "  sentence = tokenizer_pt.encode(sentence)\n",
        "  \n",
        "  attention = tf.squeeze(attention[layer], axis=0)\n",
        "  \n",
        "  for head in range(attention.shape[0]):\n",
        "    ax = fig.add_subplot(2, 4, head+1)\n",
        "    \n",
        "    # plot the attention weights\n",
        "    ax.matshow(attention[head][:-1, :], cmap='viridis')\n",
        "\n",
        "    fontdict = {'fontsize': 10}\n",
        "    \n",
        "    ax.set_xticks(range(len(sentence)+2))\n",
        "    ax.set_yticks(range(len(result)))\n",
        "    \n",
        "    ax.set_ylim(len(result)-1.5, -0.5)\n",
        "        \n",
        "    ax.set_xticklabels(\n",
        "        ['\u003cstart\u003e']+[tokenizer_pt.decode([i]) for i in sentence]+['\u003cend\u003e'], \n",
        "        fontdict=fontdict, rotation=90)\n",
        "    \n",
        "    ax.set_yticklabels([tokenizer_en.decode([i]) for i in result \n",
        "                        if i \u003c tokenizer_en.vocab_size], \n",
        "                       fontdict=fontdict)\n",
        "    \n",
        "    ax.set_xlabel('Head {}'.format(head+1))\n",
        "  \n",
        "  plt.tight_layout()\n",
        "  plt.show()"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 0,
      "metadata": {
        "colab": {},
        "colab_type": "code",
        "id": "lU2_yG_vBGza"
      },
      "outputs": [],
      "source": [
        "def translate(sentence, plot=''):\n",
        "  result, attention_weights = evaluate(sentence)\n",
        "  \n",
        "  predicted_sentence = tokenizer_en.decode([i for i in result \n",
        "                                            if i \u003c tokenizer_en.vocab_size])  \n",
        "\n",
        "  print('Input: {}'.format(sentence))\n",
        "  print('Predicted translation: {}'.format(predicted_sentence))\n",
        "  \n",
        "  if plot:\n",
        "    plot_attention_weights(attention_weights, sentence, result, plot)"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 0,
      "metadata": {
        "colab": {
          "height": 71
        },
        "colab_type": "code",
        "id": "YsxrAlvFG8SZ",
        "outputId": "f6e8f432-df39-403d-b3d8-e1b311cd5b8d"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Input: este é um problema que temos que resolver.\n",
            "Predicted translation: this is a problem that we have to solve .\n",
            "Real translation: this is a problem we have to solve .\n"
          ]
        }
      ],
      "source": [
        "translate(\"este é um problema que temos que resolver.\")\n",
        "print (\"Real translation: this is a problem we have to solve .\")"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 0,
      "metadata": {
        "colab": {
          "height": 71
        },
        "colab_type": "code",
        "id": "7EH5y_aqI4t1",
        "outputId": "e54cbe9b-3881-4d0b-c02b-fe4ba18e6b1d"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Input: os meus vizinhos ouviram sobre esta ideia.\n",
            "Predicted translation: my neighbors heard about this idea .\n",
            "Real translation: and my neighboring homes heard about this idea .\n"
          ]
        }
      ],
      "source": [
        "translate(\"os meus vizinhos ouviram sobre esta ideia.\")\n",
        "print (\"Real translation: and my neighboring homes heard about this idea .\")"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 0,
      "metadata": {
        "colab": {
          "height": 71
        },
        "colab_type": "code",
        "id": "J-hVCTSUMlkb",
        "outputId": "c7d20130-213a-4701-bdb2-b5421d312cff"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Input: vou então muito rapidamente partilhar convosco algumas histórias de algumas coisas mágicas que aconteceram.\n",
            "Predicted translation: so i 'm going to play with you some magic moments that happened , they have happening .\n",
            "Real translation: so i 'll just share with you some stories very quickly of some magical things that have happened .\n"
          ]
        }
      ],
      "source": [
        "translate(\"vou então muito rapidamente partilhar convosco algumas histórias de algumas coisas mágicas que aconteceram.\")\n",
        "print (\"Real translation: so i 'll just share with you some stories very quickly of some magical things that have happened .\")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "_1MxkSZvz0jX"
      },
      "source": [
        "You can pass different layers and attention blocks of the decoder to the `plot` parameter."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 0,
      "metadata": {
        "colab": {
          "height": 629
        },
        "colab_type": "code",
        "id": "t-kFyiOLH0xg",
        "outputId": "9bd6cdb5-d233-4f6d-dbef-7b37cd67c3f9"
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Input: este é o primeiro livro que eu fiz.\n",
            "Predicted translation: this is the first book that i did .\n"
          ]
        },
        {
          "data": {
            "image/png": "iVBORw0KGgoAAAANSUhEUgAABHQAAAIuCAYAAADeyV8dAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzs3Xl4VPXd/vF7MiHIYgQVI0ZEoChFRcGtUpdKS1gUJOIC\njyJ1+SnKIohYURRcaR8takGqFrUICG6gArJEUVutG4u2oCj7EmJYJRAgITPn9wclj1GS+ZzMOTM5\nyft1XVxXmNz5nO9MZu6c+WaShBzHcQQAAAAAAIDASEn2AgAAAAAAAOAOGzoAAAAAAAABw4YOAAAA\nAABAwLChAwAAAAAAEDBs6AAAAAAAAAQMGzoAAAAAAAABw4YOAAAAAABAwLChAwAAAAAAEDBs6AAA\nAAAAAAQMGzoAAAAAAAABk5rsBaDqW758uRYuXChJOuuss9SqVaskrwhAdUbnAEgkOgdAotA38Fp4\n1KhRo5K9CFRdEydO1JNPPqnGjRtr7969evbZZxWJRHT66acne2kAqiE6B0Ai0TkAEoW+gR9CjuM4\nyV4Eqq5u3brplVdeUd26dSVJe/bs0dVXX62ZM2cmeWUAqiM6B0Ai0TkAEoW+gR/4HTqIKRwOH/Jt\nAPADnQMgkegcAIlC38Br1fJ36DiOo/79+2vo0KFq0aJFspcTaJdffrmuvPJKdezYUZL07rvvqmfP\nnkleFVC10DneoXOAitE33qJzgIrROd6hb+CHavk7dD766CNNmzZN+/fv1/nnn5/s5QRa27Zt1aZN\nG23fvl21a9fWddddV1pCAA6gc7xD5wAVo2+8RecAFaNzvEPfwA/V8nfo3H777erZs6ceeeQRzZ49\nW6mp1fKFSL6LRqPq3r27Zs2aleylAFUaneMNOgeIjb7xDp0DxEbneIO+gV+q3e/Q2b59u1auXKkL\nL7xQ5513nnJycpK9pMBKSUnRySefrE2bNiV7KUCVRed4h84BKkbfeIvOASpG53iHvoFfqt0W61tv\nvaVLLrlE0oGfU3zqqafUpUuXJK8quLZs2aJLLrlEbdq0UZ06dUovf+aZZ5K4KiRDTk6O2rdvr3r1\n6iV7KVUKneMtOgcH0Tk/R994j87BQXTOz9E53qJvcJCXfVPtNnSmT5+uCRMmSJLatGmjLVu2KC8v\nT40bN07yyoJpwIAByV4CqoD169dr8ODBGjFihHr37p3s5VQpdI636BxIdE556Bvv0TmQ6Jzy0Dne\nom8ged831ep36BQUFOidd95Rr169Si/7+OOP1bBhQ7Vu3TqJKwOC7YknnpB04PH0+uuvJ3k1VQed\nA/iDzvk5+gbwD53zc3QO4A+v+6ZavUInPT1dLVu2LHPZr3/9ay1atChJKwqu3r17a+rUqWrbtq1C\noVDp5Y7jKBQKafHixUlcHRIpEolo7ty5euONN/Tvf/9by5cvV6tWrZK9rCqBzvEOnYOD6JxDo2+8\nRefgIDrn0Ogc79A3OMiPvqlWr9CRpOzsbM2YMSPmZQBsFixYoPnz5+uPf/yjZs2apSVLlui+++5L\n9rKqDDoH8BadUz76BvAenVM+Ogfwlh99U21eobNkyRItWbJE27dv14svvlh6+e7duxWJRJK4suBb\nuHCh1q1bp549e2r79u0qLCxUkyZNkr0sJMjrr7+uG264QZLUsWNHPfnkk/rDH/6gtLS0JK8suegc\n/9A5NRud83P0jb/onJqNzvk5Osc/9E3N5kffVJs/W75//37t2bNHkUhEhYWFpf/q16+vv/zlL8le\nXmCNGzdOEyZM0HPPPSfpwO08bNiwJK8KiVJQUKBdu3bprLPOkiTVrl1bnTp10qeffprklSUfneMP\nOqdmo3MOjb7xD51Ts9E5h0bn+IO+qdn86ptq9SNXkUhEQ4YMoWg8dNlll+nNN99Udna23nzzTUlS\nt27dNHPmzCSvDEg+Osd7dA5waPSNP+gc4NDoHO/RN/BDtfmRK0kKh8P64Ycfkr2MaqVWrVoKhUKl\nv8Brz549SV4REmXZsmUVvv+UU05J0EqqLjrHe3ROzUXnVIy+8QedU3PRORWjc7xH39RcfvZNtdrQ\nkaTWrVurX79+6ty5s+rWrVt6eVZWVhJXFVxdunTR/fffr4KCAr366qt64403dNVVVyV7WUiAP/7x\nj5Kk4uJiLV26VCeffLIk6dtvv1WbNm00derUZC6vyqBzvEXn1Fx0Tmz0jffonJqLzomNzvEWfVNz\n+dk31epHriRp+PDhh7x89OjRCV5J9fHxxx/ro48+kiSdf/75+vWvf53kFSGRhgwZon79+pUWz3ff\nfacXXnihtJhqOjrHe3ROzUbnlI++8QedU7PROeWjc7xH39RsfvRNtdvQgT92796tkpKS0v83aNAg\niatBIl122WV66623Yl4GeInOqbnoHCQDnVNz0TlINPqm5vKjb6rdj1wVFRXp9ddf14oVK1RUVFR6\nOTvJlTNt2jT95S9/0WGHHaZQKCTHcRQKhfTee+/FPbu4uFhr166VJDVr1ky1atWKeya816JFC917\n773q3r27QqGQ3n77bbVo0SLZy6oy6Bxv+dU59E1w0Dnlo2+8R+eAzikfneMtnlfBj76pdq/QGTRo\nkJo3b65Zs2apf//+mjlzppo3b64RI0ZUat7WrVs1ZswYbd68WRMmTNDKlSu1ZMkSXXnllR6vvGrK\nysrStGnTdOSRR3o697PPPtPdd9+tzMxMOY6jvLw8/elPf9LZZ5/t6XEQv6KiIk2dOlVffPGFJOns\ns89W7969Vbt27SSvrGqgc7zlR+fQN8FC55TP676R6Bw6B3RO+egcb/G8Cr70jVPNXHbZZY7jOM6l\nl17qOI7jFBcXO3369Kn0vBtvvNGZPXu2061bN8dxHGf//v2ls2uCG264wdmzZ4/nc7Ozs51Vq1aV\n/n/16tVOdna258cB/EbneMuPzqFvUF143TeOQ+fQOUD56Bxv8bwKfqh2P3KVmnrgKqWnp+u7777T\n0Ucfrdzc3ErP27Fjh7p27arnnnuudH5KSoonaw2CoUOHqlevXjr99NOVlpZWenk8O/OStH//fjVv\n3rz0/82aNdP+/fvjmilJubm5Wrdundq3b699+/appKRE9evXj3tuTbZo0SKNGzdOmzZtKvPzvl68\nPLQ6oHO85Ufn+NU3Ep3jBzqnfF73jUTnBKlz6Bt/0Dnlo3O8xfMq+NE31W5D5+qrr9bOnTs1ePBg\n3XrrrdqzZ49uv/32Ss+rW7euduzYoVAoJEn68ssvdfjhh3u13Crv/vvv169+9SuddNJJnpbtqaee\nqnvuuUeXXXaZJGnmzJk69dRT45r56quv6pVXXtHOnTv17rvv6vvvv9fIkSM1ceJEL5ZcY917770a\nPny4Tj311BrzBdcNOsdbfnSOH30j0Tl+oXPK53XfSHROUDqHvvEPnVM+OsdbPK+CL32T7JcIeW39\n+vWmy6yWLl3qXH311U67du2cq6++2snKynK++eabeJYYKFdffbUvc4uKipwXXnjB6d+/v3Pbbbc5\nL774olNUVBTXzO7duztFRUWlLw91HKfGvITTT1dccUWyl1Cl0Tne8qNz/Ogbx6Fz/ELnlM/rvnEc\nOiconUPf+IfOKR+d4y2eV8GPvql2r9AZNGiQZsyYUeay22+/XdOnT6/UvJYtW2ry5Mlas2aNHMdR\ns2bN5FSv3yNdoXPPPVevvPKKLr744jIvDYznz+tFIhHdc889evzxx3X99dd7sUxJUlpaWpk1/vhl\nbKi8c889V3/605+UlZVV5vY95ZRTkriqqoPO8ZbXneNX30h0jl/onPJ53TcSnROUzqFv/EPnlI/O\n8RbPq+BH31SbDZ1Vq1Zp5cqV2rVrl+bPn196+e7du8v8mT23rr76as2YMUMtW7YsvSw7O/tn5VZd\nzZw5U5L07LPPll4W75/XC4fD2rFjh4qLi8vckeN19tln65lnntG+ffv08ccf6+WXX1aHDh08m19T\nffXVV5KkpUuXll4WCoX00ksvJWtJVQKd4w+vO8evvpHoHL/QOT/nV99IdE5QOoe+8Q+d83N0jj94\nXgU/+qbabOisWbNGH3zwgXbt2qX333+/9PJ69erpoYcecj1vy5Ytys/P1759+/T111+X7hzv3r1b\ne/fu9WzdVd2CBQt8mZuZmanevXurQ4cOqlu3bunl8ews33nnnXr99dd10kkn6ZVXXtFFF11UI/4E\not8mTZqU7CVUSXSOP/zoHD/6RqJz/ELn/JzXfSPROQcFpXPoG//QOT9H5/iD51Xwo29CTjV7jduS\nJUvUtm3buOfMmDFD06dP19KlS3XaaaeVlk69evV0+eWXKysrK+5jVGWffPKJzjvvvDK78j8W7/Uf\nN27cIS8fMGBAXHPhva1bt2rMmDHavHmzJkyYoJUrV2rJkiWU+n/ROd7ws3Pom2Chc8rnVd9IdA6d\ng4PonPLROd7geRUO8qNvqs0rdA7KyclRy5YtVbt2bd10001avnx5md/6bZWdna3s7GzNmzdPnTp1\n8mm1VdcXX3yh8847r8yu/I/FWzwdOnRQ69at45pxqJkHf2P+j8XzMsbhw4cf8vLRo0dXembQ3H33\n3br88sv1zDPPSJJOPPFEDRkyhBOd/6JzvOFn5/jRNwfn0jneo3PK51XfSHRO0DrHj76R6ByJzqkI\nneMNnlcdQN/40zfVbkPn448/1l133aWcnBwde+yxeuqpp3TddddVqngk6fvvv9fu3btVr149jRgx\nQl9//bWGDh2q888/3+OVVy2DBg1SNBrVBRdcoK5du3o+f/To0dqyZYs6d+6sSy65pMzP0VbWG2+8\nUfp2cXGx5syZo507d8Y18ze/+U3p20VFRXr33Xd1zDHHxDVTkhYuXKi2bdsqHA6XXrZs2bIq+Qv4\nduzYoa5du+q5556TJKWmpvJnPX+EzvGGn53jR99IwemcIPWNROdUxOu+keicoHSOH30j0TkSnVMR\nOscbPK86gOdV/vRNtWurg7+B+8MPP9Qll1wS128Nlw7cmevXr6+PPvpI27Zt0+jRo/XnP//Zi6VW\neSkpKZoyZYovsydNmqRJkybpyCOP1H333adu3bpp/Pjxcc1s2LBh6b+MjAz9/ve/16effhrXzE6d\nOpX+6969u5588kl99913cc2UpBtvvFF9+/bV1q1bSy8bMWJE3HP9ULduXe3YsaN0l/7LL7/U4Ycf\nnuRVVR10jnf86hw/+kYKTucEqW8kOqciXveNROcEpXP86BuJzpHonIrQOd7heRXPqyR/+qbavULn\n4osvVufOnXXYYYdp5MiR2r59u2rXrl3peQd/vvPDDz9Uz5491apVqxrzp/UkqX379nr++efVtWtX\n1alTp/RyLwq9UaNGuu6663TuuedqwoQJGj9+vG677bZKz1u2bFnp29FoVEuXLlVhYWHc6/yxtWvX\nKi8vL+45zZo104033qg+ffrokUceUbt27ars/eruu+/WrbfeqvXr16tXr17asWOHnnrqqWQvq8qg\nc7zlV+d43TdScDonSH0j0TkV8bpvJDonKJ2TiL6R6Bw6pyw6x1s8ryqL51Xe9E21+6XIkrRz507V\nr19f4XBYe/bsUWFhoRo1alSpWcOHD1d+fr42btyot956S5FIRNddd52mT5/u8aqrpkP9ebp4/7ye\ndODPIb7zzjuaN2+eGjRooK5du6pTp0466qijKj2zT58+pW+npqYqMzNTN9xwg5o3b17pmW3btlUo\nFJLjOAqFQmrUqJHuuOOOuH/29+CfZ1y7dq2GDBminj176o033qiyf7KxpKREa9askeM4atasmWrV\nqpXsJVUpdI53/OgcP/pGCk7nBK1vJDqnIl72jUTnBKVz/Ogbic45iM4pH53jHZ5X8bxK8r5vqtWG\nzt69e7Vu3Tq1atWq9LJNmzYpHA4rIyOjUjOj0ai++eYbNWnSROnp6dqxY4fy8/PLHAPuXXXVVbrk\nkkvUuXPnSn9ugqxHjx568803JUl79uzR8OHDlZOTo6+//jrJKyvLj8dUdULnBAN9E4y+keicivh1\n29A53qNz6JzqgM4JDjonGJ3j12OqWm3o7N+/X126dNHbb7+tunXrSpJuuOEGDRkyRKeddlqlZjqO\no7ffflsbNmzQgAEDtGnTJm3dulVt2rSJa61+zV2+fLkWLlwoSTrrrLPiLseioiK9/PLLWrRokUKh\nkM4880z17t077pdb+uHFF1+s8P3XX3+965kHP08bN25U//79Pfs8HcqmTZt03HHHeT43Hn48pqqT\noHSOX30j0TkVqcqdUxX7RqJzKuLXbROkzvG6b6TgdI4ffSPROXRO+egcznEqUpXPcaSq2Tl+Paaq\n1S9FrlWrljp27Kh33nlH0oFP5Pbt2+O6gUaNGqUvv/xSs2fPliTVq1dPDzzwQKVmLVq0SJFIxPO5\nB02cOFF33nmntm3bpm3btmnYsGGaNGlSXDPvuusurVixQtdee62uueYarVq1SsOGDav0vNtvv12S\n1K1bt0P+i8fSpUs1depU5efnKz8/X9OmTdPKlStVWFhY6Z/5PPh5mjVrlqT4P09/+9vfJEkPPfSQ\nHn744TL/XnjhhUrP9Ysfj6nqpCp3jt99I1X9zvGzb6Sq3zlB6xuJzqmIX7dNUDrHj76RgtM5fvSN\nROfQOeWjc6r2OY4UvM7heZVPfeNUMytXrnR69+7tOI7jPP30087EiRPjmtejRw/HcRznsssuK72s\nW7dulZq1aNEiZ8SIEZ7PPejSSy91CgsLS/9fWFjoXHrppXHNPNSa4llnfn6+4ziOs3HjxkP+i8f1\n11/v7Nq1q/T/u3btcm644Ya4Znr9eTrnnHMcx3GcF1980Zk+ffrP/lVFXj+mqpuq2jl+943jVP3O\n8bNvHKfqd04Q+8Zx6JyK+HHbBKVz/OgbxwlO5/jRN45D5zgOnVMROqfqnuM4TvA6h+dV/jymqt1f\nuWrRooUkac2aNZo9e7ZefvnluOalpqYqEomU/mmx7du3V/pvxbdr1670N5p7OffHwuHwId+urNat\nW+vLL7/UGWecIUn66quv1K5du0rPO+aYYyRJmZmZca/tpzZt2qS0tLTS/6elpSk3NzeumV5/no46\n6ijl5uZq+vTpeumll+JaW6J4/Ziqbqpq5ySib6Sq3Tl+9o1U9TsniH0j0TkV8eO2CVLneN03UnA6\nx4++kegcic6pCJ1Tdc9xpOB1Ds+rfHpMxT2hCrriiis0YsQInXzyyTriiCPimtWnTx/1799f27Zt\n0xNPPKG5c+dq8ODBlZ73y1/+0pe5knT55ZfryiuvVMeOHSVJ7777rnr27BnXzK+++kpvvvlm6c8g\nbtq0SS1atCh9Gd/MmTNdzTv4281/yvnvbztfvHhxpdd62WWX6YorrlDHjh0VCoWUk5Oj7OzsSs+T\nvP889e7dWzfddJM2bNhQ5nNz8PrH+1vuD2XLli1x/TUCydvHVHVUVTvHz76Rqn7n+Nk3UtXvnGT0\njUTn+M3r2yYoneNH30jB6Rw/+kYKfud40TcSnVMROqdqnuNIwescnlcd4PVjqlr9UuSD9u7dq/PP\nP19jx45V+/bt4563atUqffrpp3IcR+edd17pzlpVnLts2TItWrRIjuPo7LPPVuvWreOaF2sn1q/v\nfFfWsmXLSn95mRfXX/Ln8zRy5EhPfoeJxc0336znnnsurhleP6aqmyB0jl89RudU/c5JZN9IdI7f\n/LhtgtI5XveNFKzO8aNvpGB3jhd9I9E5FaFzOMeRqu45jhS8zvH6MVUtN3QAAAAAAACqs2r1V64A\nAAAAAABqAjZ0AAAAAAAAAqbabugUFBRo7NixKigoqNIz/ZoblJl+za3JM/2a69daq4ug3OY1/T7H\n9Q/GTD/nVgdBus2DMtOvuUGZ6dfcoMz0c251EKTbvCbP9GtuUGb6NbfKz4z7D59XURs2bHBOOukk\nZ8OGDVV6pl9zgzLTr7k1eaZfc/1aa3URlNu8pt/nuP7BmOnn3OogSLd5UGb6NTcoM/2aG5SZfs6t\nDoJ0m9fkmX7NDcpMv+ZW9ZnV9hU6AAAAAAAA1RUbOgAAAAAAAAHDhg4AAAAAAEDAVNsNnXA4rMzM\nTIXD4So906+5QZnp19yaPNOvuX6ttboIym1e0+9zXP9gzPRzbnUQpNs8KDP9mhuUmX7NDcpMP+dW\nB0G6zWvyTL/mBmWmX3Or+syQ4ziOB2sCAAAAAABAgqQmewGVMfj8Edqauz1mbvKa8bq22W0xc89/\n9h/zsWsf808Vbb7AlL3p12eY505aMVZ9Wg6MmXNKIuaZ1uvvhh8z/Zpbk2f6Ndc68+jMI/XkRw97\neuxkGnzBfbbOWf20rm3eP2buuvdXm4+d1XSO5q/rYspOvi3LlHtp9hBdd8kTpmz06xWmnBScx0d1\nfMxVx5nWudWtbyRpyG9Gxuwc63mDJN35/mLzsU8//mN9tfHXpuzjA64y5Sa9ebv69HjKtoBv1thm\nfvuk+pw82DYzaj93cnO7Ws/JXD0+QiHbTOPXmwMz7S/In7xqrK5tYbv+KbVsTyNeWv6Erms1xJSN\nFhWZcnSOt/qOeUWbf9hdYWbOgzeqy/3Pm+YdvXiX+diTZg1Wn0ufNGVTttv+xPPEf92vvu0ftC2g\nuNg2c+HD6nvWCNtM46suJn72gPqeO9I2U9L+ZseYci9P66//6fW0KZvy2TJTrrqdOxyUUqeOKffS\n13/Wda2HmtcQanBEzIz1fnr0sUfoz9NvL/f9gdzQ2Zq7Xfnrtpiyplwk190CjPn8dce7Gpu/bmvM\njFNS4nKm7XZK9ky/5tbkmX7N9WutVZnXnbOnZJOr41vz+Xk/mGdas1GXn++g3Odq+mMuKDP9nFuV\nHeic2OcElowkFUc2ujq+NZ+ft9M805xdb7tOkpRvzUbsGzqS/XZ1c05mvh8bN3TczXT3Gxasc1PS\natlnGj9X0X377DPpHM9s/mG3Nhk2SywZSYq46AbJfk4S3mI/z9m8MfY34iTJKbJt6EhS/gbbzFCq\n/cdorOuUpOK6tc3Z/Hzb5yDFxf29Op47pNSta5/p4utTym7b1x03n/9yjxX3BAAAAAAAACQUGzoA\nAAAAAAABw4YOAAAAAABAwFRqQ2fXrl16+eWXS///+eefq1+/fofM3nfffVq1alXlVgcAonMAJA59\nAyCR6BwA8ajUhs7OnTs1depUU/ahhx5SixYtKnMYAJBE5wBIHPoGQCLROQDiUakNnTFjxmjDhg3K\nzs7WY489JkkqLCzUoEGD1KVLFw0bNqw026dPHy1btkzRaFTDhw9Xt27d1L17d02cONGbawCg2qNz\nACQKfQMgkegcAPGo1J8tHzp0qFasWKEZM2ZIOvDSwOXLl2v27Nlq1KiRevfurcWLF6tdu3alH/PN\nN98oPz9fM2fOlCTt3r27wmMUFBSooKDsn8YLh8Nq3LhxZZYMIAny8vIU+cmfi01PT1d6erqrOXQO\ngFiC1DcSnQMEHZ0DIJHK65yQ4ziO22G5ubnq169faYl8/vnnevbZZ/X8889LkkaNGqUzzzxT3bp1\nU58+fXT33XerSZMmuuKKK3ThhRfqoosu0vnnn69QKFTuMcaOHatx48aVuSwzM1MLFixwu1wASdKh\nQwfl5uaWuWzAgAEaOHCgqzl0DoBYgtQ3Ep0DBB2dAyCRyuucSr1C51Bq1apV+nY4HD7k7tFbb72l\njz76SFOmTNGcOXP06KOPljuvb9++ys7OLnNZOByWJF3b7Dblr9sSc0050dfUMeXKmLlZuYtiZg6q\n3Xi1ivKam7Ldmp5rnju/eKqy0nrHzDklJeaZ1uvvhh8z/Zpbk2f6Ndc6M6NpI01eM15Tpkw5ZBd4\nIaGd07y/rXMir6pj+KqYuf7ffRszc1CPFl/pzVWnm7J/vbKHKTdv8QPq1G6kKRv98mtTTgrO46M6\nPuaq40zr3CD2jVRx5/RpOVD567ZW+PHW8wZJemTFv0w5STq76Tp9sa6pKXvvVTeacvM/u19Z5z5o\nW8C/v7PN3DtZWXWutc38yeerwrkublfrOZmrx0eMJ+WlM41fbw7MtP+GhZySaeqY2suUTUmrFTsk\nad6eSepUt48pG923z5Sjc7ztnC73P69N2wsO9WGlvho3RKcPeMK0/oxPdppykjR/0ShlnTnKlA1v\n+cGUm7P+SXU5YbAp6xQVm3Jz88erc8ZtpmwoNWzKzckdqy6Z9g3A4pNsr6Z67/179NuLK74/HJTy\n4RJTrrqdOxyUUreuKTdv90R1qt/XvIaUIxvGzFjvp8ccf6Qm/uv+cjunUhs69erVU2FhoauP2bFj\nh2rVqqWOHTuqSZMmGj58eIX5yrxkEUDV4tXLeOkcALEEqW8kOgcIOjoHQCKV1zmV2tBp0KCB2rVr\np27dupW+1O/HfvySv4Nv5+fn65577lE0GlUoFNLQoUMrc2gANRCdAyBR6BsAiUTnAIhHpX/k6vHH\nHy/z/3POOaf07REjRpS+/dJLL5W+PX369MoeDkANR+cASBT6BkAi0TkAKqtSf7YcAAAAAAAAycOG\nDgAAAAAAQMCwoQMAAAAAABAwbOgAAAAAAAAEDBs6AAAAAAAAAVPpv3JVnXRr3t6cnb/Xnn9l7Qeu\n1vHK2n/GzFx1/HmuZiZVStiffDTifi1AHMLHH6fUUB1TNvXEE2Jm/jDJ/jjuMUr6w6Tfm7LNCr83\nzw0V7jMGQ7EzLvMpbVq5Gply+i9NueKj65pnlvz2TFMudcFi80xJttsr5PJ7KcZuTEmrZR952GGm\nXHSf8X7ilpv7Vays2/toAJSc9UuVNNkZO3d+G9O8u687zXzs9z6U7r7uFlM29w/7zXPX/8Ex5Zr2\ntZ+ahlJt2WhxsXmmJDkR23lGauNjzTOt2ZI8e4/Lsd2moTR3p/uhWra8U1JinmnNhmqlmWe6yTol\n9vtqTeycY+atlzZurzg0TjpuxmrTPCe9vqvjp+wpMuVWjTnSPNOabXr1UvPMyNatplz46KPNM50S\n+/OaWlsKPc/W9GdV0X22+57rbO4mU67EkIukVtxfvEIHAAAAAAAgYNjQAQAAAAAACBg2dAAAAAAA\nAAKGDR2vNaJnAAAgAElEQVQAAAAAAICAYUMHAAAAAAAgYNjQAQAAAAAACBjfN3R69+7t9yEAoBSd\nAyCR6BwAiULfAPgp3zd0pk6d6vchAKAUnQMgkegcAIlC3wD4Kd83dNq2bStJ2rJli6699lplZ2er\nW7duWrRokd+HBlAD0TkAEonOAZAo9A2An0r1+wChUEiSNGvWLF1wwQW65ZZb5DiO9u7d6/ehAdRA\ndA6ARKJzACQKfQPgp3zf0DnotNNO07333qv9+/frd7/7nVq1alVhvqCgQAUFBWUuC4fDaty4sZ/L\nBOChvLw8RSKRMpelp6crPT3d92PTOUDNksy+kegcoKYJ0jmOROcAQVde54Qcx3H8PHC7du20ePFi\nSQdeHvjBBx/opZde0k033aTLLrus3I8bO3asxo0bV+ayzMxMLViwwM/lAvBQhw4dlJubW+ayAQMG\naODAgb4dk84BaqZk9I1E5wA1VZDOcSQ6Bwi68jrH9w2dtm3basmSJdq0aZMyMjIUDoc1ceJEbdq0\nScOHDy/34yraRb622W3KX7cl5rFzoq+pY8qVMXOh2rVjX5H/mr93srLqXGvKvrLqA/Pchsdt1I5N\nx8fMXXX8eeaZ1uvvhquZKWH73JJp6pjayxaORmJnVAWufxJn+jXXOjOjaSNNXjM+Kd+98qNz+l74\niDbn7oh57DmrHleXFnfGzK3ukxn7ivzXt6OG6ORRT5iyzaZ+b8rN/fZP6nzyH0zZyMo1ppwk5URe\nVcfwVTFzKW1ifyfxoHlLHlSntvebssVH1zXl3s+5Wxd3/KMpm7pgsSkn2a+/QvZfX+emG1PSaply\n8/ZMUqe6fUzZ6L59ppzrvvnvjw3EnGu4TTOaNtLk1U8n7bvlfnRO797jlZ+/s8LjLlgwXB06jDat\nMRSxn+q99+E9+u1Fj5qyubfvN+WWXz5SraY/YMo27WvrnHm7/q5Oh//elI3u2WPKSS4ex5JSj80w\n5ebkjlWXTNuT/JI8W4+7ecz5dZ6riO18bH7xVGWlGf86k7Ef5xdNUVbta2wzJTkltvtqVe+cyvaN\nFOM859yR2rxxe4Uf7+Z+7KTXN+Ukae43o9X5lxWv/aDVD9Ux5b674n6d9PqDpmzTq5eacm66IXz0\n0abc3Pzx6pxxmykrSWrU0DZ36SPqfOq9pmzk6+9MuWr7HMj4fNXVc1VJcqKxZxrvU7E6J2G/Q+fz\nzz/X888/r9TUVNWrV09/+tOfKvy4RL5MGoA/kvEyXjoHqJmS9WMDdA5QMwXpHEeic4CgK69zfN/Q\nOfiywB49eqhHjx5+Hw5ADUfnAEgkOgdAotA3AH7K9z9bDgAAAAAAAG+xoQMAAAAAABAwbOgAAAAA\nAAAEDBs6AAAAAAAAAcOGDgAAAAAAQMCwoQMAAAAAABAwvv/Zcj+Ej2yocGHUlj36qJiZyPYfXB3f\n2V9iyvX6xcXmmfP22PKpTRuZZx7INzHlGk7dZZ7Z6F8NTLntl9Uyz5Sk8FFHmnLRggLzzFDt2qac\nU1RknomaJ5r3vSIbtpqykQ25MTMnPLTefvBRQ3TCQ5+Yok6dOuaxTu73ptzf1/3TPNOa//2J7r6X\nEP3Pd6ZcajRinpn63iJXazBzHEPGvk5JkvF6RffZ50b37bMFU8Lmma6yLj5XMW9Ty20eMLX+vVq1\n1sfunFpffGual9LQ9nX7oLQ1m025E2/Zaxt4uXTiLZtM0e8ePM02U9IKY/boxeaRkqSd/3OuKXfk\nvBXmmc7+/aZcymGHmWdas6HDbOdCpXPr2OZGdxe6mmsRPvYYX7IlG2N/ba7JNvU4QZt2xO6J9dc0\nN81rMmmluwX8YDu3b3Z9nm3eFVKz61eboptv/pVtpqStxmzGP7eZZ6pRQ3M0umKtL9maLFy/ni/Z\nyC77c+t48QodAAAAAACAgGFDBwAAAAAAIGDY0AEAAAAAAAgYNnQAAAAAAAAChg0dAAAAAACAgGFD\nBwAAAAAAIGDi2tDZtWuXXn755dL/f/755+rXr1/ciwKAQ6FzACQKfQMgkegcAJUR14bOzp07NXXq\nVK/WAgAVonMAJAp9AyCR6BwAlRHXhs6YMWO0YcMGZWdn67HHHpMkFRYWatCgQerSpYuGDRtWml22\nbJn69Omjnj176qabbtLWrVvjWzmAGofOAZAo9A2ARKJzAFRGajwfPHToUK1YsUIzZsyQdOClgcuX\nL9fs2bPVqFEj9e7dW4sXL1abNm300EMP6a9//asaNmyod955R2PGjNGjjz5a7uyCggIVFBSUuSwc\nDqtx48bxLBlAAuXl5SkSiZS5LD09Xenp6ZWaR+cAKE+Q+kaic4Cgo3MAJFJ5nRNyHMep7NDc3Fz1\n69dPM2fOlHSgeJ599lk9//zzkqRRo0bpzDPPVKtWrdSrVy+dcMIJchxH0WhUxxxzjCZMmFDu7LFj\nx2rcuHFlLsvMzNSCBQsqu1wACdahQwfl5uaWuWzAgAEaOHBgpebROQDKE6S+kegcIOjoHACJVF7n\nxPUKnUOpVatW6dvhcFiRSESO46hly5aaNm2aeU7fvn2VnZ1d5rJwOHzgfWfeq/wN22LOmLv5GXU+\nJvYvE4ts/8G8rpySaeqY2suUTUmrFTv0X/P2TFKnun1iz8xoZJ45Z80YdWl2hynbcOouU+7lX/1N\n//Pp/zNlt19mv/5z88erc8Ztpmz0J99dKM/8vZOVVedaU9YpKjLlcqKvqWPKlaaslR8z/ZprnZnR\ntJEmrxmvKVOmHHIn2UuJ6Jw+LQcqf13slzPPL56qrLTeMXPOT26TiuREXlXH8FWmbEqdOqbcvN0T\n1al+X1P2heXzTTlJyjw+T7kbY3+n7/cnXmSe6aZzFbXdrtXxMefbzJSwbaabz5Pk6ecqiH0jVdw5\n151yp/LXV9w583b9XZ0O/73pWCkNG5jXNWf9k+pywmBT1tm715Sbu+VZdW50iyn73T0nmXKrhwxV\n8yf+bMoevdgUkyR9Pmmozuljm3vkvBWmnPV8VJKc3YWmnPW8UZJCh9U25SRp7vYJ6nzkTaZs1LhW\n69dGSQo3PtaUm7PuCXVpOsSUlaSSjbmxQ7J9zc1o2kiTVz9drTqn4+jntWlHxefXy/53iE656wnT\nsZpMWmle19y8p9W5cX9T1vz4cNGPm69tY8ot+esdanvrGFM245+xn6dK0tylj6jzqfeaspIUXbHW\nlJtfNEVZta8xZZ39xaZc0s9HfJobNj5m5/7wvDo3uNG8hsiu2M+tref4sTonrg2devXqqbAw9gOr\nWbNm2rFjh7788kudccYZKikp0dq1a/WLX/yi3I+J5yWLAKoGr1/GS+cAKE+Q+kaic4Cgo3MAJFJ5\nnRPXhk6DBg3Url07devWTRdeeKEuuqjsd11DoZCkA7vLTz31lB5++GHt2rVL0WhU1113XcziAYAf\no3MAJAp9AyCR6BwAlRH3j1w9/vjjZf5/zjnnlL49YsSI0rdbtWqlyZMnx3s4ADUcnQMgUegbAIlE\n5wBwK64/Ww4AAAAAAIDEY0MHAAAAAAAgYNjQAQAAAAAACBg2dAAAAAAAAAKGDR0AAAAAAICAifuv\nXCVDZPsORbZus2WNOVeiEVtsny33f/l9sTPrNriaWWLMT2y62DxzYtMFplzXbWebZ0pSZNt2W9B4\n+0uSU1Tkag3AoTglETklJcasLeduAY4pFt2zxzzSmv39CeebZ+ZEbfl3cr8wz5SkdzbY8peedIF5\nZkq9eqZctLDQPLNactG3rrKoUHTPHtN9z3z/THH3/bvozgJTztm71/OZJ49Zbxs4xJ6d/cU7tpmS\npKH69LFnTMmu71xsHxuJmmKWc0HXWRczJSnyw05XeQvr18aSDRvNM91kXYn1Ndf4NTlIMp5bJK3b\nUnHof4fouMf/ZZoX+e+fWLeKbI5x7INc3PbWfjz6uU9tA/9qz76Tu8Q2U4/onXdfNWalTsedYc46\n+4vN2ZosUmD72uQ2a2a5T8fI8AodAAAAAACAgGFDBwAAAAAAIGDY0AEAAAAAAAgYNnQAAAAAAAAC\nhg0dAAAAAACAgGFDBwAAAAAAIGBcb+i89NJL6tq1q4YNG6b3339ff/vb38wfm5ubq1mzZrk9JIAa\njM4BkEh0DoBEonMAxCPV7QdMnTpVEyZMUGZmpiTp4osv/lkmEokoHA7/7PKNGzdq1qxZuvTSSyux\nVAA1EZ0DIJHoHACJROcAiIerDZ2RI0dqw4YNuvXWW9WzZ0+lp6dr6dKluu+++zR8+HAdccQR+vrr\nr3XKKaeoQ4cOeuSRRxQKhRQKhTR58mSNGTNGq1evVnZ2tnr06KG+ffv6db0AVAN0DoBEonMAJBKd\nAyBerjZ0HnjgAX300UeaNGmSjjjiCM2YMUOhUKj0/WvXrtVLL70kSerXr59Gjhyptm3bau/evapd\nu7aGDh2qF154Qc8880zMYxUUFKigoKDMZeFwWI0bN3azZABJlJeXp0gkUuay9PR0paenmz6ezgFg\nFW/fSHQOADs6B0Aildc5rn/kynEcOY5zyPd17ty59O127dpp9OjR6tatm7KyspSRkeHqOBMnTtS4\ncePKXJaZmakFCxZo8prx5jk50ddcHTdZM/2a68fMWo1X2Y5d4m5uTsm0SqwmxsyA3KbV9T51zTXX\nKDc3t8xlAwYM0MCBA80z6JzgzPRrrrVz5u2yz5y36++VW0wMQflcBWWmm7le9I0UrM7x6zaft/MF\nz2fOL57q+cw5G54yJq25A1KOXWHKzd1mnzl323Ou1mCR7MdcdZzpZi6d452cyKvez/TjPufDOq19\nI0k5UfvcoDzmkv04DtLM8jrH9YZORerWrVv69s0336yLL75YH3zwga666ir9/e9/dzWrb9++ys7O\nLnPZwZ8dvbbZbcpftyXmjJzoa+qYcqWr4yZjpl9z3cx8J3exKVer8Srtz2thynZtcrYpJx3YzOmY\n2ssWjkZiZ5T82zSZM/2aa52Z0bSRJq8ZrylTphxyJ9krdE7VmelmrrVvJHedc+lJF5hy83b9XZ0O\n/70pGy0sNOWk4HyugjLTOjdRfSNVrc5xc5unHH64eV3zdr6gTkfcYMo6e/eacvOLpyorrbcpG844\nxpSbs+EpdWlyuyk7+4t3TDnpwJOr6PctTdmup/z8d50cytxtz6nzUTebspEdO0y5ZPd4dZxpnUvn\nGPzoVUax5EReVcfwVbZwORtfP5vpw1rdrHNe7hJTzk3fSFKn484w5YLymKPHvHle5emGzo9t2LBB\nLVu2VMuWLfXll19q9erVOvbYY1VoPDl2+5JFAFVPIl/GS+cANVuif2yAzgFqNjoHQCKV1zmuN3RC\nxl3MiRMn6rPPPlM4HFaLFi104YUXKhQKKRwOq0ePHsrOzuYXdwGIic4BkEh0DoBEonMAxMP1hs57\n771X+nZ2dnbpS/dGjx5dJjdixIhDfrzblwcCqNnoHACJROcASCQ6B0A8UpK9AAAAAAAAALjDhg4A\nAAAAAEDAsKEDAAAAAAAQMGzoAAAAAAAABAwbOgAAAAAAAAHDhg4AAAAAAEDAuP6z5VVBKDWsUKpt\n6ZacU1IS75ISJxTyJd81s50plxO1Z2ds/MSUK82vt+Wzm5xrH2q9vRzHPhM1T0r4wD9rNpZoJL71\nxHNst1kf1tr1+DPN2ZyIPT8v92Pz3DkrbNlOmW3NMyXZOoe+QQwptWsr5bDDYucMGUmK7trl6vjW\nvPVczI2S3E2eZzs3Pcc8c36RPf/nFe+Y5z62xJa9o1l780zrOU64xYn2mZLCv2hmC27eZp+Znm7K\nOcXF5pnW+78kRYuKzNmYt6vbc/EACDc6WqlFsb/Pn3pshmleyff57haQzK+Lbo5tzHY67gxTLidq\nz0rSo2s+9zx7TzN7P1ZLbh7Pfjz2LTNjZHiFDgAAAAAAQMCwoQMAAAAAABAwbOgAAAAAAAAEDBs6\nAAAAAAAAAcOGDgAAAAAAQMBUuKGTm5urbt26eXKgDh066IcffvBkFoDqic4BkEh0DoBEoW8A+CFh\nr9AJVcM/8Qeg6qJzACQSnQMgUegbAAfF3NApKSnR3Xffre7du+v2229XUVGRJOmTTz5Rdna2unfv\nrnvvvVf79++v8HLHcSRJ+/bt00033aTXXnvNr+sEIMDoHACJROcASBT6BoDXYm7orFmzRr169dLb\nb7+tevXq6eWXX1ZxcbGGDx+up556Sm+//bZKSko0derUci+XDuwkFxYW6tZbb1X37t115ZVX+n7l\nAAQPnQMgkegcAIlC3wDwWswNneOOO05nnHGGJKl79+5atGiRVq9erSZNmuiEE06QJPXo0UMLFy4s\n93LpwE5y//791bNnT3Xv3j3mwgoKCrRx48Yy//Ly8ip9RQEkXl5e3s8exwUFBRV+DJ0DoDIq0zcS\nnQOgcoJ0jiPROUDQldc5qW4HHfyZzYMv9fup8i6XpHbt2ukf//iHLr300pjHmThxosaNG1fmsszM\nTC1YsECTVow1r3d+8VRz1ion6s/LGv2YmxN51fuZPl3/+setsx0/Yp8ZlOsfqPuUi5nXXHONcnNz\ny1w2YMAADRw40DyjKnTO5FX2zskpmWbOmmf68Xn0YZ1ScHos5dgVxmO7m1uTOyfZPeZF30hVo3Ne\nWv6Eaa3z9kwy5dzy43MZpPOx+UVTPJ952gkbTblkn+NI0tzv/tf7mT887/lM3+7/xts1SOc4UsWd\nM/HzB00z5mwaFztUCUH5mhSUmZJ0dlPj86qofWaQrn9QzkfdzCyvc2Ju6GzatElfffWVTj/9dM2e\nPVtnnnmmmjdvrk2bNmnDhg1q0qSJ3n77bZ1zzjnlXn7QoEGD9PTTT2vUqFEaNWpUhcft27evsrOz\ny1wWDoclSX1aDlT+uq0xr/T84qnKSusdM+eUlMTMHJQTfU0dU7x/WaN5rotfgpYTeVUdw1fZwhV8\nwSgz08X1n7Hxc9uxdWAzZ/empqZsdpNzTblkX3+rpN+nfJiZ0bSRJq8ZrylTpigSKXt2mp6eXuHH\nVsXOubbFQOWv2xLzeueUTFPH1F4xc4raz9hdfR5TwraZ1nVK/qzVpx6bl7vElEs5doWi37c0ZTtl\ntjXlJBdrNfaNFJzOSWaPxdM3UtXsnOtaDVH++orPc+btmaROdfvEvH6SFN23z5ST3H0uQ6m27wta\nz8ck+zmZq3XWSjPlpAObOVm1rzFl/7ziQ1PutBM26j/rjzdl72jW3pRz043hFieactKBzZzOJ91l\nC2/eZpv5w/Pq3OBGU9YpLjbl3Nz/JSn6399VE4vlds1o2kiTVz8dqHMcqeLO6XvO/dq8cXuFHz9n\n0zh1OW5AzONIUsn3+aacFJyvSVVh5qNrbM+tzm66Tl+ssz2vuqfZObFDqhrX35e5xnNSV88rjawz\nY3VOzK/Ev/jFLzRjxgzdd999OvHEE9W7d2+lpaXp0Ucf1aBBgxSJRHTaaafp6quvVq1atQ55ufR/\nO9D33nuv7rnnHj3++OO68847yz1uenq66UQMQNXVuHFj1x9D5wCojMr0jUTnAKicIJ3jSHQOEHTl\ndU6FGzqZmZmaNWvWId/3q1/9SjNmzDBf/t5775W+/eijj1a4WAA1E50DIJHoHACJQt8A8EPMX4oM\nAAAAAACAqoUNHQAAAAAAgIBhQwcAAAAAACBg2NABAAAAAAAIGDZ0AAAAAAAAAoYNHQAAAAAAgICp\n8M+WV1Vr/9BGuQUFpuzKP54VM9Ni2GfuFhAKuct7ODc18zhXI635SP4W88xQrTRTLuv2geaZ/3rN\nnk9v/YN5brj1SaZcZNm35plJ5+b+Z8yG0myfU0kK1a7t6bwgyOt/hnJ/sHXOhnvOjZlp8vC/4l3S\noTlRz7OWz7fbvFNU5GqmHMcU63p6R1Nubr49G2p3jCn3f/nWsTPL17qamVKvni3YrIl95qmtTLlQ\nbr55ZrhhQ3M22tz+tSx05ikVv79xA/OsoIgWFSm6b1/snCHjJ6ekxJes15z9xb7k7zjxPFMuJ2rP\nSra+OxC1Ze+a95Z9pv7XnL/37v9nnrqz0y9NufpvLDTPjBbvN2ett5Up62ZWQESOP1olqbVi5kpO\nzLAN/N7+tSPpfDivTnFx7pRy2GHm7L1X3WjKzf/Mng3Vsj8Hsj4HDB/f2DwztVlTc9bZs9ecDWfY\nzt+2dm5hnrnjul+Zsw1W7DHlnPPaxM4ce0SF7+cVOgAAAAAAAAHDhg4AAAAAAEDAsKEDAAAAAAAQ\nMGzoAAAAAAAABAwbOgAAAAAAAAHDhg4AAAAAAEDAuNrQ2bVrl15++eXS/3/++efq16+fqwPOmDFD\nW7bY/0Q2gJqLzgGQKPQNgESicwB4wdWGzs6dOzV16tS4Djh9+nTl5+fHNQNAzUDnAEgU+gZAItE5\nALyQ6iY8ZswYbdiwQdnZ2Wrfvr0uuugiFRYWatCgQVqxYoVOPfVUPfbYY5Kkp59+Wh988IH27dun\ntm3b6sEHH9S8efO0dOlSDRs2TIcddpheeeUVpaWl+XLFAAQfnQMgUegbAIlE5wDwgqsNnaFDh2rF\nihWaMWOGpAMvDVy+fLlmz56tRo0aqXfv3lq8eLHatWunPn36qH///pKku+66Sx988IE6deqkyZMn\na/jw4WrdunWFxyooKFBBQUGZy8LhsBo3buxmyQCSKC8vT5FIpMxl6enpSk9PN308nQPAKkh9I9E5\nQNDROQASqbzOcbWhcyht2rTRMcccI0lq1aqVcnNz1a5dO33yySd6/vnntXfvXhUUFKhly5b6zW9+\nI0lyHCfm3IkTJ2rcuHFlLsvMzNSCBQv0z5v+n3l9q+8YGjt0h3mcJCkn8qq7D0ji3Dnrn/R85vyi\nKZ7PlKR/vXan5zPn/udhz2fmRF8LxEzJn/vU/L2TzdlrrrlGubm5ZS4bMGCABg4cWOnjJ6Nz3rvj\nRvP6lj84JHbIkvkRX+5zPvWYm/uHlR/Xf27+eM9nStL8L0Z5PnPerr97P/PfD3k+c+625zyfKdlv\n0yD1jVRx50xeY7t/+va1IyBf57j+/lz/Ds2+NeU+ecU+85NXvD/HyymZ5vlMyX67VqvOmT7INCPn\nX/fZr4wLQXl8+HHuNG/PJM9nStL8z+73fqYPzwHnrHrc85mSNDfvac9nLnzR5aaBwbsfjTBny+uc\nuDd0atWqVfp2OBxWJBJRcXGxHnzwQU2fPl0ZGRkaN26cioqKXM3t27evsrOzy1wWDoclSRdM+Jty\nf7LDfCir7xiq5mP+HDPXYthn5nXlRF5Vx/BV5rzXc1MzjzPPnLP+SXU5YbApG8m3/UK1+UVTlFX7\nGlN2d/e2ppx0YDOn/ZW2B3T68h9Mubn/eVidT7M9SCLLbCcvOdHX1DHlSlPWyvXMUMg218V9NWR8\nie78vZOVVefamLmME47WpG+f1JQpUw65kxyPZHTOb8c8r9wfYnfO8geHqNX9T8TMNXn4X+Z1ubp/\nJPG+IdnvH46Lz42b6x9u1MiUm5s/Xp0zbjNloyccY8pJBzYess4eFTMXWr7WPHPerr+r0+G/t4Wb\nNbHN/PdD6tTGdlIeyrX9boa5255T56NuNmUlKdrc9rXMcptmNG6gSW8PDlTfSBV3zrXNblP+uoq/\nLvvx9civuUGZ6dfcZM8cvurf5rkdmn2rBWtONmXvvdv2DdZPXrlT511tO8er/8ZCUy6nZJo6pvYy\nZSVJ0UjsjGy3a0bTRpq8Znz16pzL/6L873dW+PE5/7pPHdsbvxnwqf0+l+zHhx/nTim1a5ty8/ZM\nUqe6fUxZSXJOa2nKzf/sfmWd+6Bt6Je250BungOGj7e96mvOqsfVpYV9s9fZs9eUm5v3tDo37m/K\nbu3cwpRb+OIdOuv6MaasJDVYsSdm5t2PRuh358d+8UHGsUdoyusDy+0cVxs69erVU2FhYcxcUVGR\nQqGQGjZsqMLCQs2bN0+dOnUqnbF79+6YM9y8ZBFA1RTvy3jpHABWQeobic4Bgo7OAZBI5XWOqw2d\nBg0aqF27durWrZsuvPBCXXTRRWXeH/rvDufhhx+uK664QpdeeqmOP/54nXbaaaWZyy+/XCNHjlSd\nOnX45V0AKkTnAEgU+gZAItE5ALzg+keuHn+87EsmzznnnNK3R4z4vx9vGTx4sAYP/vmP+2RlZSkr\nK8vtYQHUUHQOgEShbwAkEp0DIF4pyV4AAAAAAAAA3GFDBwAAAAAAIGDY0AEAAAAAAAgYNnQAAAAA\nAAAChg0dAAAAAACAgGFDBwAAAAAAIGBc/9nyqqD5qz+o3qYfYgfvkE56YXvMWDQtzdXxQ8Z8yuH1\nXc0NH3VkzEzJxlxXM635UKqLu4ITNcWK67vbL7TmI19/Z55pzaYcdph5pptsqE4dUy7csKF55uae\nrczZbTf+ypTLyNlonhk+9piYmZRG9usTBMc982+F122NHXxQOuFPC2PGHA/WdOjBLiYbs05Rkbsl\nuMx7KbLV8Dlym92yxdUanEXLYmbWPtDe1cw1d51uyqWU2Geuy4799UaSmr24yzwzVK+uOZuyfrNn\n2RTHxRUHaqAGKXt9yafutX/NsWZD4bB5ppusE42YszVRytdrlLI+9tfFlC9t59W2ZwpVhB/nTiX2\nr0tusgXN63mePfyLYvNMZ78tG61ne/7jNpuyz36OGUqxPa/cd3TIPNNNNvXjAltuc+xcaq2Ku45X\n6AAAAAAAAAQMGzoAAAAAAAABw4YOAAAAAABAwLChAwAAAAAAEDBs6AAAAAAAAARMUjZ0evfunYzD\nAqih6BwAiUTnAEgkOgeouZKyoTN16tRkHBZADUXnAEgkOgdAItE5QM2VlA2dtm3bJuOwAGooOgdA\nItE5ABKJzgFqrtRkHDQUCsXMFBQUqKCgoMxl4XBYjRs39mtZADyWl5enSCRS5rL09HSlp6cndB10\nDlD9VZW+kegcoCagcwAkUnmdE3Icx0n0Ytq1a6fFixdXmBk7dqzGjRtX5rLMzEwtWLDAz6UB8FCH\nDgch3/UAACAASURBVB2Um5tb5rIBAwZo4MCBCV0HnQNUf1WlbyQ6B6gJ6BwAiVRe51TZDZ2KdpH7\nZj2m/E0/xDzO3KWPqPOp98bMRVeti5k5aP7eycqqc60pm3J4ffPcuZufUedj+sXMRbZuM8/Mib6m\njilXmrKhVNuLteYXT1VWmu0Xr22/5mxTTpIWvniHzrp+jCnb8KVPTbmcyKvqGL7KlE2pXduUm7dn\nkjrV7WPKSlKoTp2YmbnbnlPno242z9zcs5Upt/i5O9TuZtttmpGz0ZSbs2aMujS7I2bumMyGmvjR\nfVXmu1fxdk6fk25X/rqtMY8zv2iKsmpfEzPn7C+OmTnIzeM4mTP9mutqpuE7lJK7bpCLL5HWta57\noL155nf3DdFJDz1hyqaU2GYuf2CIWo20zWz2ou3r45z1T6rLCYNtC5DkFNkeA3Pzx6tzxm0VZjKa\nHKmJCx+uMn0jxd851za7TfnrtlT48dX2cZzEmX7NTfbMP635zDy3XdP1WrzuBFP2duPGxT/fHqYL\nuj9mytaZ96UpZ/16e5D1667lds1o2kiT14yvVp1zXashyl9f8XmOm3Pg6L59ppyU/MeHHzP9eF4l\nSQU9zzLlPp06VL/q/WdT9vBXjM+rXFz/lFNtz1Xm/fshdWpznykrSSnbYj//l6Q5uWPVJdPWTxv+\np7kpt/SxITp1mO3cSZKOfysvZmbud/+rzifdFTOXkdlQE98fXm7nJOVHrix7SMkqRADeqSov46Vz\ngOqvqvSNROcANQGdAyCRyuucpPxSZMvPeQKAV+gcAIlE5wBIJDoHqLmSsqET6yWBAOAlOgdAItE5\nABKJzgFqrqRs6AAAAAAAAKDy2NABAAAAAAAIGDZ0AAAAAAAAAoYNHQAAAAAAgIBhQwcAAAAAACBg\n2NABAAAAAAAImNRkL6AyIivWKLJuiy379XeeH98pKrId25grzW/dVpnleMIpKfE823DiJ/YFvOgy\nb+U4tpgP11+S5ix735R7x5iTpE7H7bAFn5OOmmC7Te3XSCpZtyFmJqJ9LiZWfc7+/XL2FxuztlyN\nFgr5kzc+3l1nPXbizAJ7+D57/vJJC4xDh2jw/7xpSk4ffZxxphT5Pt+cdaL22z+ybXvF768fNs8K\nisivTlWk2c7Yud+0M80Lf+DTnzR281h2+7j3cmbI5fcvU2z3qeiv25hHRi9oa8qlfrnSPDPl8MNN\nuXvbX2aeOSfXnq+391vz3Hof2bLr7jzLPHODi2zGF/Zz8v2/O7PC95dkHGGeFRTRoiJF98U+d7Nk\n4N/zisNf/cwWnOoi64OUHfbzHDfZ+z6ebUyONWdH/uIc28jHpMZP2G/TiBO15VatjZ0pKazw/bxC\nBwAAAAAAIGDY0AEAAAAAAAgYNnQAAAAAAAAChg0dAAAAAACAgGFDBwAAAAAAIGB8/StX48aNU926\ndVVYWKizzjpL5513Xpn3f/7553rhhRf0zDPP+LkMADUEnQMgkegcAIlC3wA4FN//bHkoFNLAgQP9\nPgwASKJzACQWnQMgUegbAD/l+YbOX//6V7311ls67rjj1LBhQ51yyikaPny4Lr74YmVlZekf//iH\nRo8erSOPPFK//OUvvT48gBqGzgGQSHQOgEShbwDE4umGzrJlyzRnzhy9/fbbKi4u1uWXX65TTz21\n9P3FxcW6//77NWnSJDVp0kSDBw/28vAAahg6B0Ai0TkAEoW+AWDh6YbOwoUL1bFjR6WlpSktLU2/\n/e1v5ThO6ftXr16tJk2aqEmTJpKk7t2769VXXz3krIKCAhUUFJS5LBwOq3Hjxl4uGYCP8vLyFIlE\nylyWnp6u9PR0T+bTOQAO8rtvJDoHwP8J0jmOROcAQVde53j+I1ehUKj07R+XjlsTJ07UuHHjylyW\nmZmpBQsWaPKa8eY5OdHXKr2GRM70a25QZvo114+Z84unej4z5dgV5mxO1D432bfpNddco9zc3DKX\nDRgwwNOf/6Zzqs5Mv+bmRMo/Qa30zABd//mfjzQmrTmp38kf2nLF5pG+dKMk5ZRMM+US0TdSYjpn\n6tTbTDMWLBhe6eNXJDCPYx9mSvb7nBvvfXiP5zPn7XzB85mSNCd3rOcz526f4PnMbx4Z4vlMSfpg\n/t2mXJDOcSRvznOC9LWzJs+UgnPuNGfDUy7S9mz7E1ebcjkl9qP78bXBzeepvM7xdEPn7LPP1vDh\nw3XzzTeruLhY77//vnr16lX6/ubNmys3N1cbNmxQkyZNNHv27HJn9e3bV9nZ2WUuC4fDkqRrm92m\n/HVbYq4nJ/qaOqZcWclrk7iZfs0Nyky/5rqZGUq1PRTmF09VVlpv8xrmrl8YM5Ny7ApFv29pntnp\nuDNMuWTephlNG2nymvGaMmXKIXeSvULnVJ2Zrub+6AQ15szIq+oYvsoWNp7sJvv6h846NWbmoPmf\nj1TWOQ+YspdPWmDK9Tv5Qz3z7UWm7PTTjjPl3HajEzV+rkqmqWNqrwozGU0bafKqsb73jZS4zund\ne7zy83dWuJYFC4arQ4fRpnWHP1hsykkuHx/Gx7Krx7GRq5mhFPtcw33uoOiv25hy7314j3570aOm\nbOqXK025eTtfUKcjbjBlU+rXM+WkA5s5XTJtGxLO3r2m3NztE9T5yJtM2XW3nmLKffPIEP3y3idM\nWUnK+KLIlPtg/t36TdYfK8wcm3GEpk26NVDnOFL85znJ/trJTPnTuT6cO6Vm2s4d5mx4Sl2a3G7K\nStJ9/5xpyrU/cbX+tba5KTvyF+eYcm6+NkiSnNjfgbd+njKaNtLk1U+X2zmebui0bt1aXbt2Vffu\n3ZWZmamzzjqrzPvT0tL0wAMP6Oabb9aRRx6pM888U999990hZ3n9MmkAief3y3jpHAAHJeLHBugc\nAAcF6RxHonOAoCuvczz/katbbrlFt9xyS7nvv+CCCzRnzhyvDwughqJzACQSnQMgUegbALHYX4MK\nAAAAAACAKoENHQAAAAAAgIBhQwcAAAAAACBg2NABAAAAAAAIGDZ0AAAAAAAAAoYNHQAAAAAAgIBh\nQwcAAAAAACBgUpO9gMoIpaUpVLu2LWvIOUVF8S4Jh5IS9ifvRO0zQyFTrOh3bc0j3WRbP31OzMzy\nh6TWT99mnpl50V5zNnqRba2pO+2PgZQzWsfONG5gnhcEodNPVuiYY23Zs0+LmXG++E+8Swo2x/En\n76ZzrNloxD7TeuiVG3zJv3leS1Ou33Z7NuVk2/3+QLaFOZv3qK2bJWnzjIrXmlqnevWNJH1/Xm3l\n7jwsZm7jxbEzktT0gzgXVB43j2U/Hsch4/cl3Zw3uMiHP11mHmnNhurVMc8MhW3X39lrP29wk3ci\n9tvVmk3dYx7pKpsSsd9XY2VDLmYFRbhJplJT6sbMpTZrappXsmZdvEtKHONzBVdZt+c5RuGjjvQ8\nG9n+g30B1n6u5WKLwUX2ofZdTbk5m+zZlFMbmo+fcqrt3EmSlvc/3JT77pmzYmb21E+v8P28QgcA\nAAAAACBg2NABAAAAAAAIGDZ08P/Zu/PwqMq7jeP3ZAKRLYBsQkQERBEVJQiKa6UlgMoSEYUXAYvW\noiyCiBUEoaJiq6IoWrVuYXcDEZBNqbbwqshmC0plX5IY2SQQICEz5/2DktdUknlO5pyZOZPv57q4\nrjD8cs8z2z1nHiYZAAAAAADgMWzoAAAAAAAAeAwbOgAAAAAAAB7Dhg4AAAAAAIDHsKEDAAAAAADg\nMWzoAAAAAAAAeExitBdQktzcXOXm5hY7ze/3q379+lFaEQC7srOzFQgEip2WnJys5OTkKK2oZHQO\n4G1e6huJzgG8js4BEEkldY7PsiwrSmsq1YsvvqgpU6YUOy0lJUXLly+P0ooA2NW+fXtlZmYWO23w\n4MEaMmRIlFZUMjoH8DYv9Y1E5wBeR+cAiKSSOidmN3RK20Xue8Ew5ezaFzJj6bHpSqt0R8g5Kz/f\neF3Lgu+pQ0JP4/lo5kY9M8Fvnls4Wx0Se5kNW0GzzMC76uC/zWi2oOPlRnOff/yQrr/xz0azkpTT\nukLImU0Thqv52OeMM1NWHDOa+/Rvo/XrG540mk08ZPYYWLL2j+qYOi7kXL36NTR14XBP/e9VqZ3T\nfbJysg+FzFj61aNKu+KxkHPW1/8yXlfUH8dRznWjc2z1TTAQeuZUruFa/TWqG2cuPvC6Op15t/G8\n45kpZ5ll/utxdbpkjPEasp/0Gc1902WCLp0/ttSZBpVqaNFvRniqb6TSO+eGF95Q5qHcEr7zpO/H\nDtf5E8yeOxqN+1/jdcXl49jwuEGyd+zgSwz9HC9JS/NnKC2pj9FsQpVKRnNudIPdXCtgdr0uOfSm\nOlYfYDSbNeASo7l/PTdclww3P3aq88/jRnPLPx2l9r+eWOpMvXrVNWvmfXHVOf2ve0I/Zh4s9fsX\nbX1GnZs+aHRehdt3Gq8r6p3jM3s+stMNMnx5bfey+2vXMppb/OMr6lR3oNFs4MBPRnN2OjfxnBSj\nuUXbnlXnJiOMZiVJx81eryzKmqLODQYbzQbr1jSaW7J+gjpeVvrxyM9tGlQt5MzO3z2kRn8N/bry\n7KrJWtl7YImdE/UfuRozZox69+6tiy66qNjpsVyIAMzE4tt46RwgPnmpbyQ6B/A6OgdAJJXUOVHf\n0Hn88cejvQQA5QidAyBS6BsAkUTnAOUPn3IFAAAAAADgMWzoAAAAAAAAeAwbOgAAAAAAAB7Dhg4A\nAAAAAIDHsKEDAAAAAADgMWzoAAAAAAAAeEzUP7a8LGqnnGk8W++c2iFnrIICW+dfr1EdW/PRzI1q\nZoLfnVwr6HjmibrJxpln2ZhNrFHBaC6lhnlmvXoVbcxWN5pLrGz+GKhXv0bImdo2riMvqF23mvFs\nvfqhr3PL5uPSK93gVq4bnWOcGQwYZ5rm+pPN70+SVK9hLVvzjmaeFfrxXpTZwHzWV8lnPNugUum5\ndc+Ir76RpHrVqhrNpVQ3u+x2H5dx9zi2cdxgJ9eXaH4YXa9R6ONRSUqodIZ5pgvdYCfXCto4HjvH\ncK01zR/PDWzMnlkvyXg21LFT7dr2OtwLap9ldrxYN6Wm0VwgeNTW+Ue1c3zmz0fmnWM5nynJf6bZ\n9S+ZP44D1cxeq0jma/U3MF+n6X1KkpRv/nql7tlm+wVWbbP7vmTvOOdI1SpGc2dXDd1jZ1UpvXN8\nlmXjHgcAAAAAAICoi9sfucrOzlb79u2VnZ0d05lu5Xol063c8pzpVq5ba40XXrnOy/t9jsvvjUw3\nc+OBl65zr2S6leuVTLdyvZLpZm488NJ1Xp4z3cr1SqZbubGeGbcbOoFAQJmZmQoE7L1dPtKZbuV6\nJdOt3PKc6VauW2uNF165zsv7fY7L741MN3PjgZeuc69kupXrlUy3cr2S6WZuPPDSdV6eM93K9Uqm\nW7mxnhm3GzoAAAAAAADxig0dAAAAAAAAj2FDBzGvb9++ev/996O9DADlBJ0DIJLoHACRQt/EH//4\n8ePHR3sRbklKStIVV1yhpCTzjyqMRqZbuW5lPvnkk7rgggvUsGHDotPnzp2rCRMmqEePHmXOLWmt\nc+fOVfPmzdWiRYtf/NvmzZv10EMPaeLEiXr22Wc1ePBgo8yy8kqmW7lurTVeeOU699p9zunOCbXO\n0jrnww8/1OjRo/XnP/9Zs2bN0t69e9WuXTv5fD7P3FZeyXQzNx546Tr3Suap3FjqnI8//lhDhgzR\n888/r4yMDG3ZskVXXXWVqlatyu3vkcsfL7x0nXspM5ZeV/1cv379NGrUKN13332eOsZxKzeWM/nY\nctjWvn17PfHEE2rXrl3RaXPnztX777+vGTNmOH5+ffv2Vbdu3XTrrbf+4t+2b9+utWvXqmbNmho0\naJC+++47x88fQHTFUufMnj1bzZo106WXXqoDBw5o4MCB6ty5s373u985vg4A0RFLnZOTk6OkpCTV\nqFFDx44d09ixY1WzZk098sgjjq8DQOTFUt+cMn/+fL3zzjtas2aNNm7cqIQEfqgnlnHrwBU//vij\nhg4dqnbt2uk3v/mNpk2bVvRv//znP9WrVy+1adNG1157rSZMmKDCwsKif1+5cqU6d+6sNm3aaMKE\nCaWeT+PGjdWjRw+dd955rl0WALEvUp3Tq1cvtW7dWomJiapbt666du2qtWvXuna5AMSmSHVOvXr1\nVKNGDUlSMBiU3+/Xrl273LlQAGJSpPpGko4cOaKXXnpJDz30kCuXBc5jQweO+PkbvSzL0sCBA3Xh\nhRdqxYoVevvttzV16lStXLlSkuT3+zV69GitWrVK77zzjr788kvNnDlTknTw4EENHTpUDzzwgL78\n8ks1bNiQF0sAfiFWOufrr79Ws2bNnL1wAGJONDtnzZo1uvzyy9W6dWstXbpUd955p2uXE0D0RbNv\nJk2apP/5n/9RrVq13LuAcFRitBcAbxo0aJASE///7lNQUKCLLrpI0smd4p9++kn33nuvJOnss89W\nz549tXDhQl199dVFc5LUoEED3Xbbbfr666/Vr18/ff7552rWrJk6dOggSbrzzjv11ltvRfCSAYhF\nsdg5H3zwgTZu3KgnnnjCqYsJIEbEUue0bt1aq1ev1o8//qh3331X9evXd/riAoiiWOmbf/3rX1q3\nbp3Gjh2rrKwsNy4qXMCGDsrk5Zdf1pVXXln091M/6ylJWVlZysnJUdu2bSWd3FkOBoNq06aNJGnH\njh166qmntGHDBh0/flyBQKCojH788UedddZZxc6LAxcAsdY5n3zyiZ577jm9/fbbRT8OASB+xFrn\nSFLdunV17bXX6oEHHtCcOXPCvowAYkMs9I1lWXrsscf0yCOPyOfziV+z6x1s6KBMSnuQ169fX2ef\nfbaWLFly2n8fP368WrRooeeee06VKlVSRkaGli5dKunkwcqnn35abD47O9u5hQPwpFjqnL///e96\n9NFH9dprr/H7u4A4FUud83MnTpzQ7t27jecBxL5Y6JsjR45o48aNGjZsmCQpEAjIsixdd911mjx5\nslq3bl2Wi4YI4HfowHEtW7ZU1apV9de//lX5+fkKBALavHmz/vWvf0mS8vLyVLVqVVWqVElbt27V\nrFmzir73+uuv15YtW/TJJ58oEAgoIyND+/fvL/X8CgoKVFBQIMuyir4GUH5EsnO++OILjRw5Ui+8\n8IIuvvhi1y8bgNgTyc6ZP39+0QuwzMxMTZ48udin4QCIb5Hqm2rVqukf//iH5s2bp3nz5um1116T\ndPLdQpdeeqn7FxRlxoYObPP5fKX+e0JCgl555RVt2rRJv/71r3XVVVdp7NixOnLkiCTpD3/4g+bP\nn6/U1FSNGzdON910U9H31qxZU5MnT9bTTz+tK6+8Urt371arVq1KPK/MzEy1bNlSXbp0kc/nU8uW\nLdW5c2dnLiiAmBBLnfOXv/xFeXl5uueee9SqVSulpqbqnnvuceaCAogJsdQ5W7ZsUa9evdSqVSv1\n6dNHTZo0MfqkGgDeEEt9U6tWraI/Z555pnw+n2rVqlXs9/sg9vgsfkAOAAAAAADAU3iHDgAAAAAA\ngMewoQMAAAAAAOAxbOgAAAAAAAB4DBs6AAAAAAAAHsOGDgAAAAAAgMewoQMAAAAAAOAxbOgAAAAA\nAAB4DBs6AAAAAAAAHsOGDgAAAAAAgMewoQMAAAAAAOAxbOgAAAAAAAB4DBs6AAAAAAAAHsOGDgAA\nAAAAgMewoQMAAAAAAOAxbOgAAAAAAAB4DBs6AAAAAAAAHsOGDgAAAAAAgMewoQMAAAAAAOAxbOgA\nAAAAAAB4DBs6AAAAAAAAHsOGDgAAAAAAgMckRnsBiH2bNm3S6tWrJUmXX365mjdvHuUVAYhndA6A\nSKJzAEQKfQOn+cePHz8+2otA7MrIyNDzzz+v+vXr69ixY3r11VcVCAR06aWXRntpAOIQnQMgkugc\nAJFC38ANPsuyrGgvArGrS5cueuedd1S5cmVJ0tGjR3X77bdr/vz5UV4ZgHhE5wCIJDoHQKTQN3AD\nv0MHIfn9/tN+DQBuoHMARBKdAyBS6Bs4LS5/h45lWRo0aJBGjBihpk2bRns5nnbLLbeoZ8+e6tCh\ngyTpk08+UY8ePaK8KiC20DnOoXOA0tE3zqJzgNLROc6hb+CGuPwdOitWrNDs2bN14sQJXXPNNdFe\njqe1atVKLVu21IEDB5SUlKR+/foVlRCAk+gc59A5QOnoG2fROUDp6Bzn0DdwQ1z+Dp37779fPXr0\n0BNPPKGFCxcqMTEu34jkumAwqK5du2rBggXRXgoQ0+gcZ9A5QGj0jXPoHCA0OscZ9A3cEne/Q+fA\ngQPasmWLrrvuOrVr107Lli2L9pI8KyEhQRdccIGysrKivRQgZtE5zqFzgNLRN86ic4DS0TnOoW/g\nlrjbYp03b55uuukmSSd/TnHy5Mnq3LlzlFflXXv37tVNN92kli1bqlKlSkWnv/LKK1FcFaJh2bJl\nuuqqq1SlSpVoLyWm0DnOonNwCp3zS/SN8+gcnELn/BKd4yz6Bqc42Tdxt6EzZ84cvf7665Kkli1b\nau/evcrOzlb9+vWjvDJvGjx4cLSXgBiwa9cuDRs2TGPGjFHv3r2jvZyYQuc4i86BROeUhL5xHp0D\nic4pCZ3jLPoGkvN9E1e/Qyc3N1cff/yxevXqVXTaypUrVbNmTbVo0SKKKwO87bnnnpN08vH0/vvv\nR3k1sYPOAdxB5/wSfQO4h875JToHcIfTfRNX79BJTk5Ws2bNip129dVXa82aNVFakXf17t1bs2bN\nUqtWreTz+YpOtyxLPp9Pa9eujeLqEEmBQECLFy/WBx98oH/+85/atGmTmjdvHu1lxQQ6xzl0Dk6h\nc06PvnEWnYNT6JzTo3OcQ9/gFDf6Jq7eoSNJ6enpmjt3bsjTAJhZvny5li5dqqeeekoLFizQunXr\nNHbs2GgvK2bQOYCz6JyS0TeA8+icktE5gLPc6Ju4eYfOunXrtG7dOh04cEBvvfVW0elHjhxRIBCI\n4sq8b/Xq1dq5c6d69OihAwcOKC8vTw0bNoz2shAh77//vgYMGCBJ6tChg55//nn94Q9/UMWKFaO8\nsuiic9xD55RvdM4v0TfuonPKNzrnl+gc99A35ZsbfRM3H1t+4sQJHT16VIFAQHl5eUV/qlatqhde\neCHay/OsKVOm6PXXX9drr70m6eT1PHLkyCivCpGSm5urw4cP6/LLL5ckJSUlqWPHjvryyy+jvLLo\no3PcQeeUb3TO6dE37qFzyjc65/ToHHfQN+WbW30TVz9yFQgENHz4cIrGQd26ddOHH36o9PR0ffjh\nh5KkLl26aP78+VFeGRB9dI7z6Bzg9Ogbd9A5wOnROc6jb+CGuPmRK0ny+/366aefor2MuFKhQgX5\nfL6iX+B19OjRKK8IkbJx48ZS//2iiy6K0EpiF53jPDqn/KJzSkffuIPOKb/onNLROc6jb8ovN/sm\nrjZ0JKlFixYaOHCgOnXqpMqVKxednpaWFsVVeVfnzp316KOPKjc3V++++64++OAD3XbbbdFeFiLg\nqaeekiQVFBRow4YNuuCCCyRJ//73v9WyZUvNmjUrmsuLGXSOs+ic8ovOCY2+cR6dU37ROaHROc6i\nb8ovN/smrn7kSpJGjRp12tMnTpwY4ZXEj5UrV2rFihWSpGuuuUZXX311lFeESBo+fLgGDhxYVDzf\nf/+93nzzzaJiKu/oHOfROeUbnVMy+sYddE75RueUjM5xHn1TvrnRN3G3oQN3HDlyRIWFhUV/r1Gj\nRhRXg0jq1q2b5s2bF/I0wEl0TvlF5yAa6Jzyi85BpNE35ZcbfRN3P3KVn5+v999/X5s3b1Z+fn7R\n6ewkl83s2bP1wgsv6IwzzpDP55NlWfL5fPr000/Dzi4oKNCOHTskSY0bN1aFChXCzoTzmjZtqkce\neURdu3aVz+fTRx99pKZNm0Z7WTGDznGWW51D33gHnVMy+sZ5dA7onJLROc7idRXc6Ju4e4fO0KFD\n1aRJEy1YsECDBg3S/Pnz1aRJE40ZM6ZMefv27dOkSZP0448/6vXXX9eWLVu0bt069ezZ0+GVx6a0\ntDTNnj1bZ555pqO5X331lR5++GGlpKTIsixlZ2frT3/6k9q0aePo+SB8+fn5mjVrlr7++mtJUps2\nbdS7d28lJSVFeWWxgc5xlhudQ994C51TMqf7RqJz6BzQOSWjc5zF6yq40jdWnOnWrZtlWZZ18803\nW5ZlWQUFBVbfvn3LnHfXXXdZCxcutLp06WJZlmWdOHGiKLs8GDBggHX06FHHc9PT062tW7cW/X3b\ntm1Wenq64+cDuI3OcZYbnUPfIF443TeWRefQOUDJ6Bxn8boKboi7H7lKTDx5kZKTk/X999+rdu3a\nyszMLHPewYMHdeONN+q1114ryk9ISHBkrV4wYsQI9erVS5deeqkqVqxYdHo4O/OSdOLECTVp0qTo\n740bN9aJEyfCypSkzMxM7dy5U1dddZWOHz+uwsJCVa1aNezc8mzNmjWaMmWKsrKyiv28rxNvD40H\ndI6z3Ogct/pGonPcQOeUzOm+kegcL3UOfeMOOqdkdI6zeF0FN/om7jZ0br/9dh06dEjDhg3Tvffe\nq6NHj+r+++8vc17lypV18OBB+Xw+SdL69etVrVo1p5Yb8x599FFdeeWVOv/88x0t24svvlijR49W\nt27dJEnz58/XxRdfHFbmu+++q3feeUeHDh3SJ598oh9++EHjxo1TRkaGE0sutx555BGNGjVKF198\ncbl5wrWDznGWG53jRt9IdI5b6JySOd03Ep3jlc6hb9xD55SMznEWr6vgSt9E+y1CTtu1a5fRaaY2\nbNhg3X777VZqaqp1++23W2lpadZ3330XzhI95fbbb3clNz8/33rzzTetQYMGWffdd5/11ltvWfn5\n+WFldu3a1crPzy96e6hlWeXmLZxuuvXWW6O9hJhG5zjLjc5xo28si85xC51TMqf7xrLoHK90fZbH\nRQAAIABJREFUDn3jHjqnZHSOs3hdBTf6Ju7eoTN06FDNnTu32Gn333+/5syZU6a8Zs2aafr06dq+\nfbssy1Ljxo1lxdfvkS7VFVdcoXfeeUc33HBDsbcGhvPxeoFAQKNHj9Yzzzyj3/72t04sU5JUsWLF\nYmv8+dvYUHZXXHGF/vSnPyktLa3Y9XvRRRdFcVWxg85xltOd41bfSHSOW+ickjndNxKd45XOoW/c\nQ+eUjM5xFq+r4EbfxM2GztatW7VlyxYdPnxYS5cuLTr9yJEjxT5mz67bb79dc+fOVbNmzYpOS09P\n/0W5xav58+dLkl599dWi08L9eD2/36+DBw+qoKCg2B05XG3atNErr7yi48ePa+XKlZo5c6bat2/v\nWH559c0330iSNmzYUHSaz+fT1KlTo7WkmEDnuMPpznGrbyQ6xy10zi+51TcSneOVzqFv3EPn/BKd\n4w5eV8GNvombDZ3t27frs88+0+HDh/W3v/2t6PQqVapowoQJtvP27t2rnJwcHT9+XN9++23RzvGR\nI0d07Ngxx9Yd65YvX+5KbkpKinr37q327durcuXKRaeHs7P84IMP6v3339f555+vd955R9dff325\n+AhEt02bNi3aS4hJdI473OgcN/pGonPcQuf8ktN9I9E5p3ilc+gb99A5v0TnuIPXVXCjb3xWnL3H\nbd26dWrVqlXYOXPnztWcOXO0YcMGXXLJJUWlU6VKFd1yyy1KS0sL+zxi2RdffKF27doV25X/uXAv\n/5QpU057+uDBg8PKhfP27dunSZMm6ccff9Trr7+uLVu2aN26dZT6f9A5znCzc+gbb6FzSuZU30h0\nDp2DU+icktE5zuB1FU5xo2/i5h06pyxbtkzNmjVTUlKS7r77bm3atKnYb/02lZ6ervT0dC1ZskQd\nO3Z0abWx6+uvv1a7du2K7cr/XLjF0759e7Vo0SKsjNNlnvqN+T8XztsYR40addrTJ06cWOZMr3n4\n4Yd1yy236JVXXpEknXvuuRo+fDgHOv9B5zjDzc5xo29O5dI5zqNzSuZU30h0jtc6x42+kegcic4p\nDZ3jDF5XnUTfuNM3cbehs3LlSj300ENatmyZzjrrLE2ePFn9+vUrU/FI0g8//KAjR46oSpUqGjNm\njL799luNGDFC11xzjcMrjy1Dhw5VMBjUtddeqxtvvNHx/IkTJ2rv3r3q1KmTbrrppmI/R1tWH3zw\nQdHXBQUFWrRokQ4dOhRW5q9+9auir/Pz8/XJJ5+obt26YWVK0urVq9WqVSv5/f6i0zZu3BiTv4Dv\n4MGDuvHGG/Xaa69JkhITE/lYz5+hc5zhZue40TeSdzrHS30j0TmlcbpvJDrHK53jRt9IdI5E55SG\nznEGr6tO4nWVO30Td2116jdwf/7557rpppvC+q3h0sk7c9WqVbVixQrt379fEydO1LPPPuvEUmNe\nQkKCZsyY4Ur2tGnTNG3aNJ155pkaO3asunTpopdffjmszJo1axb9qVevnu688059+eWXYWV27Nix\n6E/Xrl31/PPP6/vvvw8rU5Luuusu9e/fX/v27Ss6bcyYMWHnuqFy5co6ePBg0S79+vXrVa1atSiv\nKnbQOc5xq3Pc6BvJO53jpb6R6JzSON03Ep3jlc5xo28kOkeic0pD5ziH11W8rpLc6Zu4e4fODTfc\noE6dOumMM87QuHHjdODAASUlJZU579TPd37++efq0aOHmjdvXm4+Wk+SrrrqKr3xxhu68cYbValS\npaLTnSj0OnXqqF+/frriiiv0+uuv6+WXX9Z9991X5ryNGzcWfR0MBrVhwwbl5eWFvc6f27Fjh7Kz\ns8POady4se666y717dtXTzzxhFJTU2P2fvXwww/r3nvv1a5du9SrVy8dPHhQkydPjvayYgad4yy3\nOsfpvpG80zle6huJzimN030j0Tle6ZxI9I1E59A5xdE5zuJ1VXG8rnKmb+LulyJL0qFDh1S1alX5\n/X4dPXpUeXl5qlOnTpmyRo0apZycHO3Zs0fz5s1TIBBQv379NGfOHIdXHZtO9/F04X68nnTy4xA/\n/vhjLVmyRDVq1NCNN96ojh07qlatWmXO7Nu3b9HXiYmJSklJ0YABA9SkSZMyZ7Zq1Uo+n0+WZcnn\n86lOnTp64IEHwv7Z31Mfz7hjxw4NHz5cPXr00AcffBCzH9lYWFio7du3y7IsNW7cWBUqVIj2kmIK\nneMcNzrHjb6RvNM5Xusbic4pjZN9I9E5XukcN/pGonNOoXNKRuc4h9dVvK6SnO+buNrQOXbsmHbu\n3KnmzZsXnZaVlSW/36969eqVKTMYDOq7775Tw4YNlZycrIMHDyonJ6fYecC+2267TTfddJM6depU\n5tvGy7p3764PP/xQknT06FGNGjVKy5Yt07fffhvllRXnxmMqntA53kDfeKNvJDqnNG5dN3SO8+gc\nOice0DneQed4o3PcekzF1YbOiRMn1LlzZ3300UeqXLmyJGnAgAEaPny4LrnkkjJlWpaljz76SLt3\n79bgwYOVlZWlffv2qWXLlmGt1a3cTZs2afXq1ZKkyy+/POxyzM/P18yZM7VmzRr5fD61bt1avXv3\nDvvtlm546623Sv333/72t7YzT91Oe/bs0aBBgxy7nU4nKytLDRo0cDw3HG48puKJVzrHrb6R6JzS\nxHLnxGLfSHROady6brzUOU73jeSdznGjbyQ6h84pGZ3DMU5pYvkYR4rNznHrMRVXvxS5QoUK6tCh\ngz7++GNJJ2/IAwcOhHUFjR8/XuvXr9fChQslSVWqVNEf//jHMmWtWbNGgUDA8dxTMjIy9OCDD2r/\n/v3av3+/Ro4cqWnTpoWV+dBDD2nz5s2644471KdPH23dulUjR44sc979998vSerSpctp/4Rjw4YN\nmjVrlnJycpSTk6PZs2dry5YtysvLK/PPfJ66nRYsWCAp/Nvpr3/9qyRpwoQJevzxx4v9efPNN8uc\n6xY3HlPxJJY7x+2+kWK/c9zsGyn2O8drfSPROaVx67rxSue40TeSdzrHjb6R6Bw6p2R0Tmwf40je\n6xxeV7nUN1ac2bJli9W7d2/LsizrpZdesjIyMsLK6969u2VZltWtW7ei07p06VKmrDVr1lhjxoxx\nPPeUm2++2crLyyv6e15ennXzzTeHlXm6NYWzzpycHMuyLGvPnj2n/ROO3/72t9bhw4eL/n748GFr\nwIABYWU6fTu1bdvWsizLeuutt6w5c+b84k8scvoxFW9itXPc7hvLiv3OcbNvLCv2O8eLfWNZdE5p\n3LhuvNI5bvSNZXmnc9zoG8uicyyLzikNnRO7xziW5b3O4XWVO4+puPuUq6ZNm0qStm/froULF2rm\nzJlh5SUmJioQCBR9tNiBAwfK/FnxqampRb/R3Mncn/P7/af9uqxatGih9evX67LLLpMkffPNN0pN\nTS1zXt26dSVJKSkpYa/tv2VlZalixYpFf69YsaIyMzPDynT6dqpVq5YyMzM1Z84cTZ06Nay1RYrT\nj6l4E6udE4m+kWK7c9zsGyn2O8eLfSPROaVx47rxUuc43TeSdzrHjb6R6ByJzikNnRO7xziS9zqH\n11UuPabCTohBt956q8aMGaMLLrhA1atXDyurb9++GjRokPbv36/nnntOixcv1rBhw8qcd+GFF7qS\nK0m33HKLevbsqQ4dOkiSPvnkE/Xo0SOszG+++UYffvhh0c8gZmVlqWnTpkVv45s/f76tvFO/3fy/\nWf/5bedr164t81q7deumW2+9VR06dJDP59OyZcuUnp5e5jzJ+dupd+/euvvuu7V79+5it82pyx/u\nb7k/nb1794b1aQSSs4+peBSrneNm30ix3zlu9o0U+50Tjb6R6By3OX3deKVz3OgbyTud40bfSN7v\nHCf6RqJzSkPnxOYxjuS9zuF11UlOP6bi6pcin3Ls2DFdc801evHFF3XVVVeFnbd161Z9+eWXsixL\n7dq1K9pZi8XcjRs3as2aNbIsS23atFGLFi3Cygu1E+vW/3yX1caNG4t+eZkTl19y53YaN26cI7/D\nxMQ999yj1157LawMpx9T8cYLneNWj9E5sd85kewbic5xmxvXjVc6x+m+kbzVOW70jeTtznGibyQ6\npzR0Dsc4Uuwe40je6xynH1NxuaEDAAAAAAAQz+LqU64AAAAAAADKg7jd0MnNzdWLL76o3NzcmM50\nK9crmW7lludMt3LdWmu88Mp1Xt7vc1x+b2S6mRsPvHSdeyXTrVyvZLqV65VMN3PjgZeu8/Kc6Vau\nVzLdyo35zLA/JytG7d692zr//POt3bt3x3SmW7leyXQrtzxnupXr1lrjhVeu8/J+n+PyeyPTzdx4\n4KXr3CuZbuV6JdOtXK9kupkbD7x0nZfnTLdyvZLpVm6sZ8btO3QAAAAAAADiFRs6AAAAAAAAHsOG\nDgAAAAAAgMfE7YaO3+9XSkqK/H5/TGe6leuVTLdyy3OmW7lurTVeeOU6L+/3OS6/NzLdzI0HXrrO\nvZLpVq5XMt3K9Uqmm7nxwEvXeXnOdCvXK5lu5cZ6ps+yLMuBNQEAAAAAACBCEqO9gLIYdu1Y7cs8\nEHJu+raXdEeTQSHnsv/SzPi8P+04TL9e8rzRbMrw3ca5U799Vv1ajAg556tW1Tgz46s/qv8V44xm\nj7RKMZr74JV71GPga0azlZb9y2hOkqZvfVF3NB1iNhwMmGVuf1l3NL7PeA2eyfT5zHIN7/+SJMN9\nXdO11k45U8+veNzsvD1g2DVjzDrH8PqZuupb4/NOqPOZgnt/ZTTb/6pLjOambX5RfZsZPt585m/k\nnPb9ZPU9//6Qc1ZhoXFmNO/HdsVEP8RRpmluvPWNJD3wm8e1L+tgqTOmxw2StHX8ecbnvfK2e3X1\nu38xmm30QdBobuY7g/Q/t79kNFthc6bRXMbaJ9U/dbTRrBLND3czVj2m/m0fNRs27DI7aw0ePmw0\nZ6fHrYDZ7STZ7FwXMv21zjSay/h6gvq3GWu8huAhs48GNnkeq51ypp77m9nxtVeYHOfY6Xlfovm7\nDuzclzP+1+y1hZ1jp35tWxjNxdtzZ7xmupUbzcxQxzme3NDZl3lAOTv3Gs2azGUdrWPr/LOO/mQ0\nl7hrn63cHIP5hOoFtjJ/3BP6Ragk5Z5dzTjzh71mT4qVDW+jU0xvU9MNHVuZNkQ903BDx1aujTfq\nuXH5Y53TnaOA2QsWu/M5O+sbR+bsNOwnGxs6prlW4Qmbmd65H0e9H+Is083cWLYv66DRMYHJjCTt\nOVLX1vnvOWL2PH/GDzaej384ZDRXYfd+80zDWV+FCsaZkvmxk3XCvMtM12q68SCZ97gVML+dTuZG\nrx/8x2xkGt5OkhQ8YHbsLtl4fowjpsc5prejz8Ym6slcw+vczvGT8bGT+evA8v7c6ZVMt3JjNTNu\nf4cOAAAAAABAvGJDBwAAAAAAwGPY0AEAAAAAAPCYMm3oHD58WDNnziz6+6pVqzRw4MDTzo4dO1Zb\nt24t2+oAQHQOgMihbwBEEp0DIBxl2tA5dOiQZs2aZTQ7YcIENW3atCxnAwCS6BwAkUPfAIgkOgdA\nOMq0oTNp0iTt3r1b6enpevrppyVJeXl5Gjp0qDp37qyRI0cWzfbt21cbN25UMBjUqFGj1KVLF3Xt\n2lUZGRnOXAIAcY/OARAp9A2ASKJzAISjTB9bPmLECG3evFlz586VdPKtgZs2bdLChQtVp04d9e7d\nW2vXrlVqamrR93z33XfKycnR/PnzJUlHjhxxYPkAygM6B0Ck0DcAIonOARCOMm3onE7Lli1Vt25d\nSVLz5s2VmZlZrHgaNmyoPXv26PHHH9f111+va665ptS83Nxc5ebmFjvN7/erfv36Ti0ZgMuys7MV\nCASKnZacnKzk5OSws+kcAD/npb6R6BzA6+gcAJFUUuc4tqFToUKFoq/9fv9pz2zevHlasWKFZsyY\noUWLFunJJ58sMS8jI0NTpkwpdlpKSoqWL1+u6dteMl7XssC7xrOmvksfbzZoc7N8yRHn3y65KPNF\nxzNXfvCg45mStKxwtvOZwffKbabkzv3fzlr79OmjzMzMYqcNHjxYQ4YMCXsdEe2c7S8br8uN2zLh\nrM1Gc0sLzDOXFpj9vL5dS/NnOJ4Z7ftxtHPLc6adXC/1jVR650z99lmjdblx3CBJOwf8wWxwgHnm\np5+PLttiSrF476uOZ0rSoqwpoYdscmOtbvW4K53rQubibPPXA3aYPo/FU+eYHue41fNu3JdNj52W\nBc0z4/G5Mx4z3cqNdmZJnVOmDZ0qVaooLy/P1vccPHhQFSpUUIcOHdSwYUONGjWq1Pn+/fsrPT29\n2Gl+v1+SdEeTQcrZuTfkeS4LvKsO/ttCzu15v0XImVO+Sx+vC+eON5o9p+9249wlRzLUsWr/kHMJ\n1c13/RdlvqjOKWZPKrlXNjKaW/nBg7q6xzNGs5XnrTaak05u5nRI7GU2HAyEntHJB0iHhJ7Ga/BM\nps9nlmt4/5ckWZZZpuFa6zWqo+nbX9aMGTNOexBiV9Q7p/F9Zp1jeP0syVofcuaUhLM2K/hDM6PZ\nTudcbjS3tGCW0ir2NluAz/xXrS3Nn6G0pD4h56zCE8aZ0bwf2xUT/RBHmaa5XuwbqfTO6ddihHJ2\n7Sv1+02PGyRp0wvmxzk7B/xBjd78k9HseRlmz8effj5av76+9Bebp1T4dqfR3OK9r6pTnd8bzfp+\n9uI4lEVZU9S5wWCjWeuEWZfZWWvwUG7oIdnrcStgdjtJNjvXhUx/3TpGc4uzX1Kn+oOM1xA88JPR\nnMnzWL1GtTXt+8lx1Tkmxzl2et6XaP4S0859efEus9cWdo6dOja4zGgu3p474zXTrdxoZoY6zinT\nhk6NGjWUmpqqLl266LrrrtP1119f7N99P3vBeerrnJwcjR49WsFgUD6fTyNGjCj1PJx6yyKA6HHq\nbbx0DoBQvNQ3Ep0DeB2dAyCSSuqcMv/I1TPPFH+XRtu2bYu+HjNmTNHXU6dOLfp6zpw5ZT07AOUc\nnQMgUugbAJFE5wAoqzJ9bDkAAAAAAACihw0dAAAAAAAAj2FDBwAAAAAAwGPY0AEAAAAAAPAYNnQA\nAAAAAAA8psyfchVN/hrV5T9caDZ7Zs2QM/X+cob5maebzxe0u9A813A+KSvXVqZVo5qteSf5mzV2\nZT7w7y1lWU7pEvzuzFpBs7mffSRlyLOvXNnx2eDRo8aZRmu1cXm8wN/gLCUGk4xmExueHXLmV79r\nY3zef58v/ep3vzOarXyBeT8kXNDUbPCHvcaZkpSQXDXkjK9qFVuZieeEvk4laUefhsaZe0ZfZTR3\n9lNfGWdKstcPTmea9o3kzmPURqbPb349+RJLP1TxJbpwnUebP0EyuY4Mr8fmLxwxP+8B5vMHW9Yw\njj3cyKxDa+0wf47zGT7HWVUrGWdKklXT7KOd97WtbZy57+YLjOZqTltlnGkFLbNBn83/vzWc9yU4\n/5i3Dpk/j9mZtdWPoWbtZJVXLt3nTlgBo7kkG7OeYue523TWMuwRxCzeoQMAAAAAAOAxbOgAAAAA\nAAB4DBs6AAAAAAAAHsOGDgAAAAAAgMewoQMAAAAAAOAxbOgAAAAAAAB4DBs6AAAAAAAAHuP6hk7v\n3r3dPgsAKELnAIgkOgdApNA3AP6b6xs6s2bNcvssAKAInQMgkugcAJFC3wD4b65v6LRq1UqStHfv\nXt1xxx1KT09Xly5dtGbNGrfPGkA5ROcAiCQ6B0Ck0DcA/lui22fg8/kkSQsWLNC1116r3//+97Is\nS8eOHSv1+3Jzc5Wbm1vsNL/fr/r167u2VgDOys7OViAQKHZacnKykpOTXTtPOgcon6LRNxKdA5RX\nXjrGkegcwOtK6hyfZVmWm2ecmpqqtWvXavXq1XrkkUfUpUsX/eY3v1Hz5s1L/b4XX3xRU6ZMKXZa\nSkqKli9f7uZyATioffv2yszMLHba4MGDNWTIENfOk84Byqdo9I1E5wDllZeOcSQ6B/C6kjonYhs6\n0sm3B3722WeaOnWq7r77bnXr1q3E7yttF7l/6mjl7N4f8rwX731Vner8PuTc8VaNQ86c8tnSh/Wr\ntKeMZn02rtm/LXtYN3QInZuUlRty5pTFG59Up4tGG80ebn6m0dzKDx7U1T2eMZqt9m3o2+iUxd9N\nVKcLRxnNBv69xWhuWfA9dUjoabaABL9ZZuFsdUjsZZYpSVYwdGbgXXXw32YcmVC5stHcksNvq2O1\nO41mg0ePGs2ZrrVeozqavu2lqPzvlSudc/Vj+nHPwZDnvWjnc+rcaHjIubyW5v8b9vf5D+m6Ln82\nmq28w6wfFv/rcXW6ZIzZAn7YazYn8871Va1inLlo+yR1bvyA0eyOPg2N5r57fLguHPOc0ezZT31l\nNCeVoR+czjToG8l+57iR6fObde7SgllKq1j6LwGt16i2pm1+MWrv0HGjc/pdMlI5u0p/Dl1y6E11\nrD7AbJGNU8zmJC1ZP0EdLxtrNHuwZQ2juVVTR6htv2eNZmt9vsdozrRvJcmqWsloTrJ37LSvbW2j\nudVvPaDLfzvJaLbmtFVGc270jd1cX4LPaM7kcVyUmWj2wwNLjk5Tx8p9jWYlySosNJqL9c4pa99I\npXfOHY3vU87O0p/v7RxX+ypUNJqTpKX5M5SW1Mdodv6OL4zmkupvU352E6PZm1NaG83Zel1hyHam\nz+wxZ+s52XArICYufxRzo5lZr1EdTd/+comd4/qPXJ3aL8rKylK9evXUs2dPHT16VN9++22pxROJ\ngzAA7orG23jpHKB8itaPDdA5QPnkpWMcic4BvK6kzonY79BZtWqV3njjDSUmJqpKlSr605/+5PZZ\nAyiH6BwAkUTnAIgU+gbAf3N9Q+fU2wK7d++u7t27u312AMo5OgdAJNE5ACKFvgHw31z/2HIAAAAA\nAAA4iw0dAAAAAAAAj2FDBwAAAAAAwGPY0AEAAAAAAPAYNnQAAAAAAAA8xvVPuXJD4OBPCuw/YDZr\nMFfhE7Os/59fY2veVOKnoXODifZusuDm7UZzn3wy0zDxQX0yZYrRZNeUNoaZJwX+vcXWvKOCAXdm\nTVmW+dnn5bkya8xkrTYujxdYuYcV/OmQ0azJXNLHmbbOP+nj1UZzARvXe2Djv43mPsr82jhTkub+\nc0nIma4Nr7SVWbg7y2ju7Cd3mwU+PlxnP/m/ttZgzI1+iHLnuJFpFRY6NmsVunD9RJlVcEJWQYHB\nXOgZSdK/zY4FinIN58/MqW6ceeaybUZzu/qeZ5y56/ZzjOaCFYwjJUk7u9cxmmv0XrZxZu2VPxjN\nBRN8xpk+w1k7j7eTizB7TFlB80jjNfjM/6/ZCpgvgM4pnS/RL5/BawyTGUmSjfuxnfnfDLzXaO4f\n88xn/Wnm96MTaZcbzSUePmGcabW71Hj2QIvKxrP7B5gda9V6a5VxphL8ZnN2ysFn475iox+M12qH\nnUw3jt1KwDt0AAAAAAAAPIYNHQAAAAAAAI9hQwcAAAAAAMBj2NABAAAAAADwGDZ0AAAAAAAAPIYN\nHQAAAAAAAI8Ja0Pn8OHDmjnz/z/uetWqVRo4cGDYiwKA06FzAEQKfQMgkugcAGUR1obOoUOHNGvW\nLKfWAgClonMARAp9AyCS6BwAZRHWhs6kSZO0e/dupaen6+mnn5Yk5eXlaejQoercubNGjhxZNLtx\n40b17dtXPXr00N133619+/aFt3IA5Q6dAyBS6BsAkUTnACiLxHC+ecSIEdq8ebPmzp0r6eRbAzdt\n2qSFCxeqTp066t27t9auXauWLVtqwoQJ+stf/qKaNWvq448/1qRJk/Tkk0+WmJ2bm6vc3Nxip/n9\nftWvXz+cJQOIoOzsbAUCgWKnJScnKzk5uUx5dA6AknipbyQ6B/A6OgdAJJXUOT7LsqyyhmZmZmrg\nwIGaP3++pJPF8+qrr+qNN96QJI0fP16tW7dW8+bN1atXL51zzjmyLEvBYFB169bV66+/XmL2iy++\nqClTphQ7LSUlRcuXLy/rcgFEWPv27ZWZmVnstMGDB2vIkCFlyqNzAJTES30j0TmA19E5ACKppM4J\n6x06p1OhQoWir/1+vwKBgCzLUrNmzTR79mzjnP79+ys9Pb3YaX6/X5J0R+P7lLNzb8iMZcH31CGh\np/F5mnAj006uL9H8JltaMEtpFXsbzc7b+YXRXKX6O3Qs+1yj2a4pbYzmJO/cVl7JdCvXNLNeozqa\nvv1lzZgx47Q7yU6KROf0u2SkcnbtD5mx5NCb6lh9QMi54JEjxutaFnhXHfy3mQ0b7s/buW98lPm1\n2XnLvB+6NrzSOHNZ4Wx1SOxlNhwMhJ5RfD7m4jHTNNeLfSOV3jl9LximnF2l/wjF0mPTlVbpDlvn\nacJObkKN6kZzi7NfUqf6g4xmd/c9z2hu45+H66KHnjOaDVYIPXPKd08M14WPmOU2ei/baG7x939W\np/MfMpoN7thtNGfnGM8qLDSak6LfD74KFY3mlubPUFpSH+M1WCcKjObKbec0G6KcnSE6x8Z9Tv/J\nNWGnc46lXWo09495I3Vtt6eNZv35QaO5zxb/Qb/q9Cej2cTDJ4zmPlk5Rr+5+nGjWUk60KKy0dza\nvz6g1N9NMpqt9dYqozlbx2OW2XVq6xhXknxmvy3G1loN2c40OCZ16nVVWBs6VapUUV5eXsi5xo0b\n6+DBg1q/fr0uu+wyFRYWaseOHTrvvJKftMN5yyKA2OD023jpHAAl8VLfSHQO4HV0DoBIKqlzwtrQ\nqVGjhlJTU9WlSxddd911uv7664v9u8/nk3Ryd3ny5Ml6/PHHdfjwYQWDQfXr1y9k8QDAz9E5ACKF\nvgEQSXQOgLII+0eunnnmmWJ/b9u2bdHXY8aMKfq6efPmmj59erhnB6Cco3MARAp9AyCS6BwAdoX1\nseUAAAAAAACIPDZ0AAAAAAAAPIYNHQAAAAAAAI9hQwcAAAAAAMBj2NABAAAAAADwmLARyRBHAAAg\nAElEQVQ/5QqRZQUCrswn+SoYZ9qZBeJB8EiegocPm80aztliWc5nGup27tXGs0vzzeaX7Fllaw1L\n9qwxmuvU+ArjTF9SktGclZ9vnAk4xSooMLrvuXX/NM0N7j9gnGk6m/Ki2eNdfzaf7fXNdrNMSdJw\njez/vtHk7GVpxqnBGlWM5qzCQuNMO7NeYRWecGUWpbMKA0b3J+P7nM37pmnnVFq01jjTdNb0eECS\nklZ+ZzS37c2mxpk7h5kf41X5zHhUwYqmgzZeW9qZNWXnGNeK8lrdyHQA79ABAAAAAADwGDZ0AAAA\nAAAAPIYNHQAAAAAAAI9hQwcAAAAAAMBj2NABAAAAAADwGDZ0AAAAAAAAPMb2hs7UqVN14403auTI\nkfrb3/6mv/71r8bfm5mZqQULFtg9SwDlGJ0DIJLoHACRROcACEei3W+YNWuWXn/9daWkpEiSbrjh\nhl/MBAIB+f3+X5y+Z88eLViwQDfffHMZlgqgPKJzAEQSnQMgkugcAOGwtaEzbtw47d69W/fee696\n9Oih5ORkbdiwQWPHjtWoUaNUvXp1ffvtt7rooovUvn17PfHEE/L5fPL5fJo+fbomTZqkbdu2KT09\nXd27d1f//v3dulwA4gCdAyCS6BwAkUTnAAiXrQ2dP/7xj1qxYoWmTZum6tWra+7cufL5fEX/vmPH\nDk2dOlWSNHDgQI0bN06tWrXSsWPHlJSUpBEjRujNN9/UK6+84uylABCX6BwAkUTnAIgkOgdAuGz/\nyJVlWbIs67T/1qlTp6KvU1NTNXHiRHXp0kVpaWmqV6+erfPJzc1Vbm5usdP8fr/q169vd8kAoiQ7\nO1uBQKDYacnJyUpOTjbOoHMAmHCibyQ6B4AZOgdAJJXUObY3dEpTuXLloq/vuece3XDDDfrss890\n22236e2337aVlZGRoSlTphQ7LSUlRcuXL9f07S8b5ywLvmfrfKOV6VbussC7jmcmnLXZ7LyD9nK9\nclt5JdOtXDuZffr0UWZmZrHTBg8erCFDhjiyFjontjIlaWn+DMczTTtn6THzzKXHppdxNaXzym3l\nlUw7uW73jRR7nRPt69yOpQWznM906XE84PyVZnOrzDOXrhpXxtWUzEu3v1eOcSU6pzReus+50TlL\nDr/teObmnmPNh3uaj65/6QGzwZfMM73SDW7lRjuzpM5xdEPn53bv3q1mzZqpWbNmWr9+vbZt26az\nzjpLeXl5Rt/fv39/paenFzvt1C8Du6PxfcrZuTdkxrLge+qQYOOeb8CNTFu5P3sbZsjMwLvq4L/N\naHZJ5jqjuYSzNiv4QzOj2Y4NLjOak7xzW3kl061c08x6jepo+vaXNWPGjNPuJLuBznEn01ehonHu\n0vwZSkvqE3Ju8U7zV0F2OqdT4yuM5pYem660SncYzVr5+UZzUvRvq3jLNM2NRt9I0e+cWHju8CWa\nHUYuLZiltIq9zRZwml/8etpMG4/jXt9sNztvndzMefP7q41mZ9+RZjS3dNU4pbX9o9GstXqD0Vws\n3P6uZBoe59o5xpUklfAOmF/k0jklioX7nBud40tKMppbcvhtdax2p9HstjebGs1t7jlWzd6bYDQr\nSVU+q2I0t/6lB3TZoElGs3X+8oXRXNS7Icq5sfy6yvaGjs+waDMyMvTVV1/J7/eradOmuu666+Tz\n+eT3+9W9e3elp6eX+ou7yvKWRQCxxYm38dI5AEw49WMDdA4AE3QOgEgqqXNsb+h8+umnRV+np6cX\n7fROnDix2NyYMWNO+/123x4IoHyjcwBEEp0DIJLoHADhSIj2AgAAAAAAAGAPGzoAAAAAAAAew4YO\nAAAAAACAx7ChAwAAAAAA4DFs6AAAAAAAAHgMGzoAAAAAAAAeY/tjy2OB/5yzleivYjSb2OTckDOF\n23aEt6AS+BLtXb12540y/X6juU6N2hrNLc03nz3nqwpGc/8/b3ab7rnBMs5MqFzZaC54PN84Uwlm\n16ktNjL91ZPNZ2vWNJoLHDxonFkuJfjNbyOTuWAgvPV4XKdzLjeeXVpgPj9nxz+Mc+dsNZtNP6ed\ncaYkbn8gTFZBgeOzsy8+xzhzQIH5/OJd0w1Tx2nxR2azHVNaGWZK8vnM5izz4yaUU6bHOabHQi49\nz/kqVXJ8NnjkiHFm8OhRo7nGd2wyC+xpY1bSU9+bHuc8oDcfet5o8g+vXGl8/lHvHNPztzMbB/3I\nO3QAAAAAAAA8hg0dAAAAAAAAj2FDBwAAAAAAwGPY0AEAAAAAAPAYNnQAAAAAAAA8ptQNnczMTHXp\n0sWRM2rfvr1++uknR7IAxCc6B0Ak0TkAIoW+AeCGiL1Dx2fnY8YAIEx0DoBIonMARAp9A+CUkBs6\nhYWFevjhh9W1a1fdf//9ys/PlyR98cUXSk9PV9euXfXII4/oxIkTpZ5u/ecz3o8fP667775b7733\nnluXCYCH0TkAIonOARAp9A0Ap4Xc0Nm+fbt69eqljz76SFWqVNHMmTNVUFCgUaNGafLkyfroo49U\nWFioWbNmlXi6dHInOS8vT/fee6+6du2qnj17un7hAHgPnQMgkugcAJFC3wBwWsgNnQYNGuiyyy6T\nJHXt2lVr1qzRtm3b1LBhQ51zzjmSpO7du2v16tUlni6d3EkeNGiQevTooa5du4ZcWG5urvbs2VPs\nT3Z2dpkvKIDIy87O/sXjODc3t9TvoXMAlEVZ+kaicwCUjZeOcSQ6B/C6kjon0W7QqZ/ZPPVWv/9W\n0umSlJqaqr///e+6+eabQ55PRkaGpkyZUuy0lJQULV++XBmfjzZe76ItTxvPmloWdOdtjUsLZnkj\nM3+G45mS9Eabt80Gj5hnLjmSUaa1lGZZ4WxPZErS4v2vOZ5p5/7fp08fZWZmFjtt8ODBGjJkiHFG\nLHTO9K0vGq/XlfuHC53jWo+50A9u9FjVBjuN5pYV2sstz7e/W/cp01wn+kaKkc7Z/rLRWqN9ndvh\nxuN4WeBdxzMld9aacNZmo7llAfNMty6/V/oh2pffS8c4kjPHOW4dr7px/1hy6E3HM924z7n1uiq1\n0S6juWh3jmvPYx5ZqxOvq0Ju6GRlZembb77RpZdeqoULF6p169Zq0qSJsrKytHv3bjVs2FAfffSR\n2rZtW+LppwwdOlQvvfSSxo8fr/Hjx5d6vv3791d6enqx0/x+/8l/u/5J/Zh5MOSFXrTlaXU+b2TI\nucJtO0LOnLIs+J46JJi9rdGXaL5ftrRgltIq9jaedzzTZ/b7sZfmz1BaUh+j2YYrKpidt05u5tz1\n9Z1Gs3tuKPnJ7eeWHMlQx6r9jWaDx/ON5pYVzlaHxF5Gs6bsZvqrJxvNLd7/mjrVusdoNnAw9ONJ\nMr//12tUR9O3v6wZM2YoECj+TJGcXPr6Y7Fz7mg6RDk794a83Ma3ZdD82dNO57iR6atQ0TjXuB+s\noHmmjR6bs+N/jeaqNtipI1mNjGbTz2lnNCfF5+0fzUzT3HD6RorRzml8X8jOieZ1forpcY6dx7EV\nMHt8LAu8qw7+24xmff+5Xk3YWeviXauN5hLO2qzgD82MZjumtDKas3P5VcqmwC9yo90Phr/s19bl\nl4yvA7c7J1p9I4V/nGPreNWl57mEatWM5pYcelMdqw8wmg0eMfufYludk2j2GsjO6ypJeur7fxjN\npTbapbU7zzGa/UOTK43m3Ogc233jRj+4tVYHM0N1Tshn4vPOO09z587V2LFjde6556p3796qWLGi\nnnzySQ0dOlSBQECXXHKJbr/9dlWoUOG0p0v/vwP9yCOPaPTo0XrmmWf04IMPlni+ycnJRgdiAGJX\n/fr1bX8PnQOgLMrSNxKdA6BsvHSMI9E5gNeV1DmlbuikpKRowYIFp/23K6+8UnPnzjU+/dNPPy36\n+sknnyx1sQDKJzoHQCTROQAihb4B4Aazn7MBAAAAAABAzGBDBwAAAAAAwGPY0AEAAAAAAPAYNnQA\nAAAAAAA8hg0dAAAAAAAAj2FDBwAAAAAAwGNK/djyWBU4s4oK8wuNZgtrVws9tC3MBZXACgQcn/cl\nVrC3CJ/Znp2/9pnGkaaze35vnqm10p7fNzIa9TUsMI71NWxgNJeQZP5QSLi4mfns/lyjucT69Ywz\n85udZTxbcGljo7mkrVWMMxMbnh1yxt+gpnGeF/guOU++2nXMZlu3CDljrd5gcwE+sznLspdrctYV\n7XWOybyVn1/W5ZSqa9+BRnPLPzWfTaqbaWsNiXVrh5zZl9bEVuZP/doZzdVekW2cmdjkXLPBYNA8\n89xzjGeVb97jifVL7zx/XRvPNR5xvMOlOrY39PPHsW5tjfIqzVsV7pJOy85xjumsz+83zjSd9SUl\nGWfamb/p6m5Gc4u2ms8e7W7+HH+0u9ntX22FvQNdfx3D5zsbzw+JKWbHY1b1qsaZ/gvNj8esijaO\n8y4r/Xk8oX4N4yyv8PkTjB5Ppo85K2jvNZCp4OHDrswaMzzOsk6YP8fZmR19eWejucV7zWelg8bn\nb+rgnWbHLXZn915t9vpfkr5/7XKjuRaP7jbODHU88nOFP+SYDZoc44eY4R06AAAAAAAAHsOGDgAA\nAAAAgMewoQMAAAAAAOAxbOgAAAAAAAB4DBs6AAAAAAAAHsOGDgAAAAAAgMfY2tA5fPiwZs6cWfT3\nVatWaeBAs49+PWXu3Lnau3evre8BUD7ROQAihb4BEEl0DgAn2NrQOXTokGbNmhXWGc6ZM0c5OYaf\nyw6gXKNzAEQKfQMgkugcAE5ItDM8adIk7d69W+np6brqqqt0/fXXKy8vT0OHDtXmzZt18cUX6+mn\nn5YkvfTSS/rss890/PhxtWrVSo899piWLFmiDRs2aOTIkTrjjDP0zjvvqGLFiq5cMADeR+cAiBT6\nBkAk0TkAnGBrQ2fEiBHavHmz5s6dK+nkWwM3bdqkhQsXqk6dOurdu7fWrl2r1NRU9e3bV4MGDZIk\nPfTQQ/rss8/UsWNHTZ8+XaNGjVKLFi1KPa/c3Fzl5uYWO83v96t+/fp2lgwgirKzsxUIBIqdlpyc\nrOTkZKPvp3MAmPJS30h0DuB1dA6ASCqpc2xt6JxOy5YtVbduXUlS8+bNlZmZqdTUVH3xxRd64403\ndOzYMeXm5qpZs2b61a9+JUmyLCtkbkZGhqZMmVLstJSUFC1fvlzT595vvL5lXz5qfmFMM4PvOZ4p\nScsC7zqeuTR/huOZizJfdDxTkpas/aPjmYu/m+h45pL1ExzPXLR7suOZkrR8+SjHMxftfM54tk+f\nPsrMzCx22uDBgzVkyJAyn380Omfah+ads/QrFzrHhW5wq8eWHH7b8cylBeG9Jf10ln/q/GNDkhZl\nTQk9ZNPXbz/geOaiLU87n7ntWcczJfPnHC/1jVR657z/2u+NMlbMHWl+YWxwox9cOcZxoRskd3ps\n0dZnHM9c+cGDjmdK0uKclx3PdOM4Z/GGJxzPlMyPR+Opc6Z9b3b7uPG6QnKpc8pxpiQt3vuq45lu\n9PjqN50/xpGknXc9ZDZ4l3mmG6+B7VynJXVO2Bs6FSpUKPra7/crEAiooKBAjz32mObMmaN69epp\nypQpys/Pt5Xbv39/paenFzvN7/dLku5In6yc7EMhM5Z9+ag6XPlY6DNb9S/jdS0LvqcOCT3Nhn0+\n89zAu+rgvy10ZGKFkDOnLM2fobSkPkaz/tpnGs0tynxRnVPMnqiC9cwypZNPnh1TxxnN+o4VGM0t\n/m6iOl1o9qLNSjJ7KCxZP0EdLxtrNCtJCftzQ84s2j1ZnRuabxjkNzvLaG758lFq395sQytp649G\nc4t2PqfOjYaHnKt7dk1lrHxUM2bMOO1Ocjii0Tl9u5t1ztKvHlXaFaE7x1q9wXhdpt1wMtjsoM5O\njyVUqWJ23jr5IqhjtTtDzlk2bpulBbOUVrG30WzhtS2N5pZ/Okrtf2342PguM/TQfyzKmqLODQaH\nnNuX1sQ48+u3H1CbOycZzdZekW00t2jL0+p8nuFGQDBolrntWXVuMsIsU5LyzXrc5Dmn7tlnKuOr\nP3qqb6TSO+fWe17VD3tLf/5YMXekrkk325irNG+V8brcOM6x02O+/1wHodjpBl9SktGcZN5jkpRQ\nt7bR3KKtz6hzU7PNl9zLzJ7jV37woK7uYbZJVG3FNqM56eRmTqd69xnN+iqaHZPaOc6xqlc1mlu8\n4Ql1uvgRo1lJsioaHucZHI/Wq19DUxcOj6vO6Xv+/crZua/U77fzusI6Ydbxks3OKeeZ/lpmr60W\n731VneqY/cdA4MBBozk7PX6w/5VGc6vffECXDzA7xpGkvVcXGs3tvOshNXrjz0azLR7dbTRn5zWw\nJBX+EPp3W5lep/Ua1dH0bS+V2Dm2NnSqVKmivLy8kHP5+fny+XyqWbOm8vLytGTJEnXs2LEo48iR\nIyEz7LxlEUBsCvdtvHQOAFNe6huJzgG8js4BEEkldY6tDZ0aNWooNTVVXbp00XXXXafrr7++2L/7\n/vM/NdWqVdOtt96qm2++WWeffbYuueSSoplbbrlF48aNU6VKlfjlXQBKRecAiBT6BkAk0TkAnGD7\nR66eeab42zvbtm1b9PWYMWOKvh42bJiGDRv2i+9PS0tTWlqa3bMFUE7ROQAihb4BEEl0DoBwJUR7\nAQAAAAAAALCHDR0AAAAAAACPYUMHAAAAAADAY9jQAQAAAAAA8Bg2dAAAAAAAADyGDR0AAAAAAACP\nsf2x5bEg4fvdSti1z2x247aQM8EEv80FGM5bQXu5JpEnClyZD+w1uz7tzPrPSDLOlKSEQ3lGc4U7\ndhlnBr7fajZ4xSXGmcHKFY1nfYFqRnNWDbM5SdrRxfz8TWfPf/KocaZ11GD2WCXjPE/YuEXaafgY\nWfddyJGEqlVtnb3pfPDwYVu5Rpl5Zo9LW/M2O9cKWkZzFTMPGWeazhb+kGOcaTq/94pzbWXuvcLs\nuaTWGvPOtSqZzebXM7+vHm9Sx3i2wt+/MZ4N9ZwTrGwc5RlV1u1R1T0HQs5V/XKHUV4gzPWUyGfj\n/wUNZ63CQuNI01nTDjkleOy40Zzvp1zjTMtwtsqCbOPMKgvWGc35atU0zpQknz96/9/73QPJrsye\nN838UVBYvfR+LKxmfhzmFdaJE0avGey+DnGcz+f8rGWvH6Lp/9q79+io6nON489kQlI4GCRcQhIi\ninJXT7kEQaVWIBD0nLCQpQtv4NIKyMUq2nKpAhaKWutheYiAerQERBR6BKxIQMHKEu9HvFcuURBi\nDEiFcJGEzOzzBwtqukzm3Th7Znby/fyFMw/P/u3JzJudHzMxXHE4+lk352/Mphyy/wzsJts8wz5z\n3WT9jnfoAAAAAAAA+AwbOgAAAAAAAD7Dhg4AAAAAAIDPsKEDAAAAAADgM2zoAAAAAAAA+ExcNnSu\nvfbaeBwWQAPFzAEQS8wcALHEzAEarrhs6CxbtiwehwXQQDFzAMQSMwdALDFzgIYrLhs63bt3j8dh\nATRQzBwAscTMARBLzByg4YrLhk4gEIjHYQE0UMwcALHEzAEQS8wcoOFKjvcCalNRUaGKiooatwWD\nQWVmZsZpRQDcKisrUygUqnFbWlqa0tLS4rSi2jFzAH/z07yRmDmA3zFzAMRSbTMn4DiOE+vF9OjR\nQ++//36dmXnz5qmwsLDGbdnZ2dq4caOXSwMQRf3791dpaWmN2yZMmKCJEyfGdB3MHKD+S5R5IzFz\ngIaAmQMglmqbOXHZ0Onevbu2bNlSZ6auXeSR3e5W+VffRjzOukOLNPiMmyLmwt8fi5g56eXqZ5WX\nPMIWdsL23tBy5QWvMXTav1wvh1coL+lqUzaQbHuz1vqqZRqUYvtN+sGcbFNOktaW/ElDzr3blK3e\n+ZUpZ35MJemiC2ydb9yrvItn2TolBQ9VRswUfzxb+RfcY+7cdnNzU+6LO+9S+7kPm7Id52wz5Yr3\nPab8VmMi5jJyWqjo/TkJ869XP3Xm3Nhhosp3RZ451tdHoHHjiJmT1h18SoOb3WzKhg8dMuXczAY3\nzL1JQXuni5kbPO9sU6748weU33mKKRvaVmLKSfbz3/7oRebOnbfdrbMX/MmU7Tz/gCnnZuZUZjQ1\n5V59ZYouH/iAKStJjTZ9aMpZXlMZ7VpqyfZ5CTNvpJ8+c0bl3qvyPf+o8+8Xlz2q/MzxpvWEyvea\ncpLL+WB8Lbu6dgqHImfkzTollzOnme25Vbz/ceW3GG3Khg8fMeXWVy7VoNTrTdlgC9t1gySt/bpQ\nQ7Im2MJB2+O6dvcjGpLza1P2s/uyTLldN09Wu6ceNGUl6bwltufVhlenacDlc+rMZGQ00zPPjq9X\nM+eGc8apfNe+Ov9+3K8dJMn40TJXPwMYf7by4vzddgYapZhybuaDc7zKlHOz1iPDbdc5b6y4Wxdf\nbbvGkaSjo2zXOR/8x2z9/EXbdU7mmIOm3NrSeRqSbd+srf6mPGLG+jzNaNdKT3/xaK0zJy4fubJ8\nzjOR37IIwCZR3sbLzAHqv0SZNxIzB2gImDkAYqm2mROXX4oc6S2BABBNzBwAscTMARBLzByg4YrL\nhg4AAAAAAABOHxs6AAAAAAAAPsOGDgAAAAAAgM+woQMAAAAAAOAzbOgAAAAAAAD4DBs6AAAAAAAA\nPpMc7wWcDqeqUk5lpS1ryYVD7hbgNm/lON70Wg4dsp+TNRsuK3e1BnPezeNkzIaT7XubbrJfzUwx\n5b405iSp0+RvbME7pU6P2rLxe+b5g1MdklNdbcxGzjmHDrk6fthlvqFySo2vDZfZaDt7tYvvIbfZ\n8+lP2GeuNbt/4G5zZ8pbfzdnw2H71HEiZJ2wuco3nGPH5Bz9PnLOkPGUmwffmg0E7J3GbHJ2pr3T\nRd45ZrsWPVFqu+QOpDQyV1qzTtMm5k43+c/vSTd3/v0Ptse068y9tsKbpa6zjVlJzhH7a6XRZ3vq\nvv/QUXMXosyDnwH8xDle5Uk22pq+sMWT7Ib/3mxMztabPZ82JQvK+5iPX12+z5w1P/8suQgZ3qED\nAAAAAADgM2zoAAAAAAAA+AwbOgAAAAAAAD7Dhg4AAAAAAIDPsKEDAAAAAADgM57+X64KCwvVpEkT\nHTlyRL169VLfvn1r3P/OO+/oqaee0sKFC71cBoAGgpkDIJaYOQBihXkD4Md4/r8tDwQCmjhxoteH\nAQBJzBwAscXMARArzBsA/yrqGzoLFizQ6tWrlZWVpebNm6tbt26aOnWqLr/8cg0aNEibNm3S/fff\nr/T0dHXp0iXahwfQwDBzAMQSMwdArDBvAEQS1Q2dTz/9VGvXrtULL7ygqqoqXXXVVTr//PNP3V9V\nVaXp06dryZIlysnJ0R133BHNwwNoYJg5AGKJmQMgVpg3ACyiuqHz3nvvKS8vTykpKUpJSdGAAQPk\nOM6p+7/44gvl5OQoJydHklRQUKDly5f/aFdFRYUqKipq3BYMBpWZmRnNJQPwUFlZmUKhUI3b0tLS\nlJaWFpV+Zg6Ak7yeNxIzB8A/+ekaR2LmAH5X28yJ+keuAoHAqT//cOi4VVRUpMLCwhq3ZWdna+PG\njVqyfZ65Z33VstNeQ21eDq+IeqdXvZ50hmr/ZvFTrDu6JOqdXpz/hk2/i3rn1uHT7eHh9ujaHQ+5\nX0wExfseM2evv/56lZaW1rhtwoQJUf38dyxmztNfzjf3+OZ17Kc5Vv1s1DvXHVoU9U7Jm/P/W/Hk\nqHc+0+cJW/CwvXPd4aLTW0wE1q9/LOaNFJuZs/hj2+xed/Cp0z5+Xfxy7eDV9cjaXXOj3llcbv8+\nYuXVHCve9seod+4cOcUWHGnvXPvFw6e3mAisXys/XeNI0bnO8dW1QwPu9KrXi871lUuj3ilJjTN3\nmnIvV9s7vbgedfOY1jZzorqhk5ubq6lTp2r06NGqqqrSq6++qhEjRpy6v3379iotLdXu3buVk5Oj\nNWvW1No1atQoDRs2rMZtwWBQknRjh4kq3/VtxPWsr1qmQSnXRsw51fav5MvhFcpLutqcj2evq84f\nfMOoszO0XHnBa0zZpNRU27F1YjNncJMbTdnwsWOmnJvzD1/6c1Nuw6bfacAv/mDKStKeX4ciZrYO\nn65O//t7c2f7yYdMubU7HtKQ835jyjoHKyKHdGIzJ7/VmIi5jJwWKnp/jpYuXfqjO8nREquZc8M5\n41S+a1/E9cT9dRzHTle9SUF7Z/WzykseETkoKanxz0y5dYcWafAZN5my4SNHTDnJfv7HB/Uyd/6t\neLJ+mf+gKZs1c4cp90yfJ3TdW7easvsHVply6w4XaXDTUaasJIWPVZpylq9/RrtWerpknufzRord\nzBl5wW9U/tX+Otey7uBTGtzsZtO6w4ds3zek+F87WLnpTG6bbe5du2uuhrS705R1jM/j4vL5ys8Y\nZ+s8etSUczPHApmtTTnpxGZOfsffmrKf35Nuyu0cOUVnL37AlO0yM/L3WunEZs6Q9neZspLkHPne\nlLN8rTJy0lX03mxfXeNIP/06J+7XDnTGtddNZ6BRiim3vnKpBqVeb17D6p2bTbnGmTv1fdnZpmxB\nTh9Tzs31qCQpHPlnQOtjmtGulZ7+cn6tMyeqGzpdu3bVFVdcoYKCAmVnZ6tXr5oXrSkpKbrvvvs0\nevRopaenq2fPntq2bduPdkX7bdIAYs/rt/EycwCcFIuPDTBzAJzkp2sciZkD+F1tMyfqH7kaM2aM\nxoyp/V/w+/Xrp7Vr10b7sAAaKGYOgFhi5gCIFeYNgEiS4r0AAAAAAAAAuMOGDgAAAAAAgM+woQMA\nAAAAAOAzbOgAAAAAAAD4DBs6AAAAAAAAPsOGDgAAAAAAgM+woQMAAAAAAOAzyfFewOlIOvNMJR0O\n27It0iNmwgcOujp+IDXVlHOOV7vqVVIwciYcctdp5ThRz4arjrtagjUfPO8cc6c1e7yJ/aUQcpHN\nnm/YMx0uZc9vZO6sPCvyc9ptNhBqbu483rVd5EybZuY+PwhmtVFy2Pa6T85pG1akR2AAAAyZSURB\nVDFTvXvPT12Sv7mdY8Z8+MgRe6U1GwiYO635n23Z5arSmt9/W6atcIs9u2ecfTbsGfdzczZYaY6q\nfPxFdXelp9nLfOJgXkd9921FxNx3Q7uZ+potfdvdAqzPew+uHVwxdrqduV7M6NC+fbagi5kTPnrU\nFtzxpblTkkLGfIebv7IVjpQ63PyBKbpzSt2v9xrZ6yJ/vz2pyTf259++/zyvzvuTW9S/mWO9zrFc\n40hc5/iKm+scYzap8c/MlW6yw/4935Qr3mvPljzY0Xz8kgd7m7PNttty347uGzHTKMLM4R06AAAA\nAAAAPsOGDgAAAAAAgM+woQMAAAAAAOAzbOgAAAAAAAD4DBs6AAAAAAAAPsOGDgAAAAAAgM+woQMA\nAAAAAOAzbOgAAAAAAAD4THK8F1CbiooKVVRU1LgtGAwqMzMzTisC4FZZWZlCoVCN29LS0pSWlhan\nFdWOmQP4m5/mjcTMAfyOmQMglmqbOQHHcZw4ralO8+bNU2FhYY3bsrOztXHjxjitCIBb/fv3V2lp\naY3bJkyYoIkTJ8ZpRbVj5gD+5qd5IzFzAL9j5gCIpdpmTsJu6NS1izwq916V7/lHxI7iskeVnzk+\nYi584KB5Xeu/f1qDGt9gyjrHq829L1c/q7zkEZGD4VDkzMnO8ArlJV1tzke9Mylo77Wev6Rg+7NM\nueJtf1R+x9+assfatzDl/lY8Wb/Mf9CUlaSk4+GImY0bpqr/gPvNnTK+YjdunKr+/W29gZCtdMNr\n0zTgsjkRcxltmumZ58b76l+v6pw5l/xee/d8F7Fj7a65GtLuzoi56t17zOuK++s4zr1x7wwE7L2h\n5coLXhMxF2zZ0txZXD5f+RnjTFkny9a7bsvvNbj7dFN2T15zU+7TP96pbr+da8pKUrDSlvvokTt1\n4a/r7s1KT1PxjFt8NW+kumfO0ElPqOzbilr+5gnvLL5LvUc+bDpWs6Vvm9dlfR5LkoyXkHF/Hce5\n14uZ48XXSfLmOs/NNd6eKReZcn+ffae63GOfOU2+sT0G//c/k9TzV/9VZyazRZpefPBX9WrmWK5z\nrNc4Uv28zvFLp+teD2ZO8IwzTLniA08q/8xbTFlJUkojW+/ehcpvPdaU3Ta5oyn3xV2T1P7humfD\nDzXbHjmzZeEkdR8buTOzRZpe+kPtMydhP3KVyAMRgI2f3sbLzAH8zU/zRmLmAH7HzAEQS7XNnLj/\nUuR77rlHn376abyXAaCBYOYAiBXmDYBYYuYADU/c36Eze/bseC8BQAPCzAEQK8wbALHEzAEanri/\nQwcAAAAAAADusKEDAAAAAADgM2zoAAAAAAAA+AwbOgAAAAAAAD7Dhg4AAAAAAIDPxP3/cnU6Wmae\nac5mtE2PmAmnNXJ1/IyzWppyTnXIXW+7VpFDYQ86XTJ3JgU96Q1mN7d3GrOVGWnmzjYusoHjjimX\nkdHM3ClbpaveQNhemtEmcmfLVmeY+/ygpeGcT2rdNvJzLpRU6er4cX0dJ0BvXDsDgaj3BtPtM0yS\nMnIifx+TJKe1i++Nxu+j4eb2eZflIhusMkeVlV53b0azpvYyn2jd3HZOmS1tj/kZLl9D5teH4+J7\nh09mg1e9XswcL75OrnpdXOdZO8Nnupg5LrKNjddjkpTZou5e6+vTT6zXOZZrHKn+Xuf4pdNVrwcz\nJ9j03+ydZ7UwZ9XI/jN7Ro6t90iafY5ku8ieYTytSPNGklqfWffMCTiOyykPAAAAAACAuKq3H7kq\nKytT//79VVZWltCdXvX6pdOr3obc6VWvV2utL/zymDf05xzn749OL3vrAz895n7p9KrXL51e9fql\n08ve+sBPj3lD7vSq1y+dXvUmeme93dAJhUIqLS1VKOTuI0qx7vSq1y+dXvU25E6ver1aa33hl8e8\noT/nOH9/dHrZWx/46TH3S6dXvX7p9KrXL51e9tYHfnrMG3KnV71+6fSqN9E76+2GDgAAAAAAQH3F\nhg4AAAAAAIDPsKGDhHfjjTfqL3/5S7yXAaCBYOYAiCVmDoBYYd7UP8GZM2fOjPcivJKamqqLLrpI\nqampCd3pVa9XnXPmzFGnTp2Uk5Nz6vaVK1dq1qxZGj58+Gn31rbWlStXqnPnzurateuP3jd8+HA9\n+eSTWrhwoR5//HH16NFD2dnZvnpMG/Jzqj7xy2Put+dctGdOpHXWNXMkaffu3Zo8ebKmT5+uRYsW\n6cCBA7r44ot987XyS6eXvfWBnx5zv3Se7E2kmTNjxgxNmjRJjz/+uB577DEVFhZq+fLlGjNmDF9/\nn5x/feGnx9xPnYn0c5UkzZ07V5MnT9b8+fP1+uuv64ILLlB6erpvHlOvehO60wFcuvzyy5033nij\nxm3PP/+8c91113lyvBtuuMFZsWLFj97n5XEBJIZEmjlVVVXOwIEDnUWLFjnHjh1zKisrna1bt3qy\nDgDxkUgz519NmTLFmTZtmifrABB7iTRv1qxZ4/Tr18/Zs2ePEw6HnYcfftgZNmyYJ+tA9PCRK3hi\n7969uv3229W3b18NHDhQS5YsOXXfRx99pBEjRig3N1f9+vXTrFmzVF1dfer+zZs3a8iQIcrNzdWs\nWbPisXwAPhOrmbNy5UplZGRo1KhRSk1NVUpKijp27OjZeQFITPG4zjl69KjWrVunYcOGRfVcACS2\nWM2b0tJS9ezZU9nZ2QoEAiooKFBJSYln54XoYEMHUeE4To0/jx07Vl26dNHrr7+uRYsWafHixdq8\nebMkKRgMatq0aXrnnXf03HPP6a233tIzzzwjSfruu+90++23a9KkSXrrrbeUk5Oj999/v85jf/bZ\nZ+rbt6/y8/M1f/58hcNh704UQEKI18z54IMPlJWVpVtvvVV9+vTRyJEjtW3bNm9PFkDcxfM656T1\n69erRYsW6tWrV/RPEEDCiNe8ufLKK7Vr1y7t3LlTx48f1/PPP69f/OIX3p4sfrLkeC8A/jR+/Hgl\nJ//z6VNVVaVu3bpJOrFTfODAAd12222SpLZt2+rqq6/WmjVrdMkll5zKSVJWVpauueYavfvuuxo5\ncqRee+01dejQQXl5eZKkm266SX/+859rXUdubq5efPFFZWdna/v27brjjjuUnJys0aNHe3HaAOIk\nUWZOeXm53n77bS1cuFB9+vRRUVGRxo0bp+Li4hrrA+BviTJzfmjVqlUaOnRotE4RQIJIlHnTqlUr\n9ezZU/n5+UpOTlabNm1UVFTkxSkjirj6xGmZP3+++vTpc+q/V65ceeo3pn/99dcqLy9X7969JZ3Y\nWQ6Hw8rNzZUk7dy5Uw888IA++eQTHTt2TKFQ6NQw2rt3r9q0aVPjWJmZmbWuo23btqf+3KFDB40f\nP15PPvkkGzpAPZMoMyc1NVU9e/bUpZdeKkm65ZZbtGDBApWUlKhTp07RO2EAcZUoM+eksrIyvfvu\nu5o9e3ZUzg9A4kiUeVNYWKhPPvlEmzZtUsuWLbV69WqNHDlSL730Er8oPIGxoYPT8sO3Av6rzMxM\ntW3bVuvWrfvR+2fOnKmuXbtq7ty5aty4sYqKirR+/XpJUuvWrbVhw4Ya+bKysqitDYA/JcrM6dSp\nk7Zs2XIaZwDATxJl5py0evVq9ejRo8Y/ZAGoHxJl3mzdulVXXHGFWrduLUkaNmyY5syZox07dtR4\nJxASC79DB1F34YUXqmnTpnriiSdUWVmpUCik7du36+OPP5YkHTlyRE2bNlXjxo1VUlKiZcuWnfq7\nl112mXbs2KFXXnlFoVBIRUVF2r9/f63H2rRp06n7S0pKtGDBAg0cONDbEwSQUGI5cwoKCvThhx/q\nzTffVDgc1qJFi5Senq5zzz3X8/MEkBhiOXNOWrVqla666irPzglAYorlvDn//PNVXFys/fv3y3Ec\nrVq1StXV1WrXrp3n54nTx4YOXAsEAnXen5SUpIULF+rzzz/XgAEDdPHFF+vee+/V4cOHJUmTJ0/W\nX//6V/Xo0UMzZszQlVdeeervNm/eXI888ogeeugh9enTR7t371b37t1rPdabb76pgoICde/eXWPH\njtXgwYM1ZsyY6JwogISQSDPnnHPO0UMPPaQZM2aod+/e2rhxoxYsWMDvzwHqkUSaOdKJX8ZeXl6u\nwYMH//STA5BQEmnejB49Wp07d9bQoUOVm5urxYsXq7CwUE2bNo3OycITAYfPpwAAAAAAAPgK79AB\nAAAAAADwGTZ0AAAAAAAAfIYNHQAAAAAAAJ9hQwcAAAAAAMBn2NABAAAAAADwGTZ0AAAAAAAAfIYN\nHQAAAAAAAJ9hQwcAAAAAAMBn2NABAAAAAADwmf8HF5dT1QETKRQAAAAASUVORK5CYII=\n",
            "text/plain": [
              "\u003cmatplotlib.figure.Figure at 0x7fa1bc8a2310\u003e"
            ]
          },
          "metadata": {
            "tags": []
          },
          "output_type": "display_data"
        },
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Real translation: this is the first book i've ever done.\n"
          ]
        }
      ],
      "source": [
        "translate(\"este é o primeiro livro que eu fiz.\", plot='decoder_layer4_block2')\n",
        "print (\"Real translation: this is the first book i've ever done.\")"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "colab_type": "text",
        "id": "RqQ1fIsLwkGE"
      },
      "source": [
        "## Summary\n",
        "\n",
        "In this tutorial, you learned about positional encoding, multi-head attention, the importance of masking and how to create a transformer.\n",
        "\n",
        "Try using a different dataset to train the transformer. You can also create the base transformer or transformer XL by changing the hyperparameters above. You can also use the layers defined here to create [BERT](https://arxiv.org/abs/1810.04805) and train state of the art models. Futhermore, you can implement beam search to get better predictions."
      ]
    }
  ],
  "metadata": {
    "accelerator": "GPU",
    "colab": {
      "collapsed_sections": [
        "s_qNSzzyaCbD"
      ],
      "name": "transformer.ipynb",
      "provenance": [],
      "toc_visible": true,
      "version": "0.3.2"
    },
    "kernelspec": {
      "display_name": "Python 3",
      "name": "python3"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}
